Pedablogy: Musings on the Art & Craft of Teaching

This is a fairly long post that was driven by conversations with a wide variety of colleagues. All errors are my own.

Cognitive science has discovered a great deal about how learning works, and by extension, what leads to effective teaching. From the evidence available, relatively few higher education faculty, at least at 4-year institutions, incorporate these findings into their teaching. Many colleges and universities claim to value quality teaching, often stating that it is their number one priority (e.g. for tenure and promotion). Why then do they not perceive, or more importantly, act on the disconnect between what is known about effective teaching and how higher ed instructors actually teach? Why do colleges and universities feel no compulsion to seriously address that disconnect, when the literature shows that teaching is a learnable skill; that good teachers are made, not born?

At 4-year colleges and universities, the assumption is that all (or nearly all faculty) are effective teachers.  By implication, instructors must use effective teaching practices (e.g. not over-reliance on lecture), and thus, teaching quality is assumed to be good (enough).

Contrast this with the fact that few new Ph.D.s received serious training in learning science, which implies that, at least initially, new faculty are unlikely to be excellent teachers.  Much as faculty expect incoming students to know how to do college-level writing, or at least to pick it up on their own, incoming faculty  are expected to pick up teaching talents through experience, with occasional programs offered by their school, though the majority of faculty do not participate in these programs. 

Some say there is no evidence of a problem.  [Inside joke: How many free market economists does it take to screw in a lightbulb? Answer below.] There is limited information on how faculty actually teach.  In my discipline, economics, the most recent evidence suggests that lecture remains the dominant pedagogy.  This appears to be also true in the natural sciences and some other disciplines, such as history. I’m not trying to criticize specific disciplines; just to note that lecture is still widespread.

Do institutions really want to know how effective their teachers are? Are they prepared to deal with the consequences? Suppose an administration undertakes a poll to determine how prominent effective teaching practices are among its faculty.  Suppose they discover that effective teaching practices are not the norm.  What does the administration do then? Maybe ignorance is the smarter course.

Perhaps administrations don’t see the benefit of improved teaching.  If most teachers do a good job so that the room for improvement is small, while the cost of teaching development is large, the perceived marginal benefits may not be worth the cost. Has any institution explored this question seriously? Or is it another lightbulb joke? What institutional metrics could show the effects of improved teaching?  Consider student success metrics, like completion rates.  For students who started full time in 2012 at 4-year institutions, the national average 6-year completion rate was 66%.  Note that this was before Covid.  For my state, Virginia, the figure was nearly 80%. These are very noisy statistics, meaning the margin for error is high. Would improved teaching effectiveness significantly increase completion rates?  Would it reduce the number of students dropping out for academic reasons or transferring?  It seems like the potential benefit could be large (e.g. up to 20%).  Even 5% improvement would be significant, especially for those students involved.

The problem is how we assess teaching effectiveness!

We don’t know how to measure teaching quality with precision. Education is a complex process with multiple inputs and multiple outputs, making it challenging to determine the effects of a specific innovation, be it a new pedagogy or the effectiveness of an instructor in a single course.

There is more than one type of effective teaching. Some teachers are better in one context than in others. Some teachers are better with introductory students; some are better with advanced students. Some shine at helping students in trouble. Others excel at making sure all students reach a particular standard. Some are better with small groups; some with large groups, etc. Do these faculty get assigned to where their teaching is most effective?  Sometimes yes; sometimes no, since a specific teacher’s comparative advantage isn’t always the most important determinant of their teaching assignments. 

Serious teaching assessment is time and resource intensive. By contrast, most assessment gets dumbed down to the lowest common denominator that we can quantify.  At most schools there is over-reliance on student course evaluations, which can be better or worse, and which have known biases.  As one department chair told me “Course evals will only give you low-res[olution] assessment.” How many chairs and deans consider the biases in their evaluation of teachers? How many consider whether differences in student course evaluation scores are statistically significant?  

We don’t have a good quantitative measure of teaching effectiveness.  How about qualitative measures? Department chairs play a critical role in teaching evaluation. Chairs who take teaching evaluation seriously can identify teachers’ strengths and weaknesses and they can differentiate between good and poor teachers. A casual survey of department chairs suggests that good teachers can judge good teaching, because they’ve internalized the multiple dimensions. What would a complex, multi-environmental assessment look like? It would involve reviewing course designs, multiple class visits, grade distributions and well-designed course evaluation results, ideally by more than one reviewer.  What would we hope to learn from this assessment?  What would its value-added be over student course evaluation data? Ideally, it should identify directions for improving teaching effectiveness, based on evidence-based instructional practices.

But consider a school which does not use professional chairs, that is, chairs selected for their management and leadership ability.  It is not uncommon for schools to rotates the position around each department, regardless of one’s ability to chair.  Ideally, chairs would act based on what’s best for the department, but chairs are human beings.  If you are an associate professor, would you feel comfortable being critical of a more senior colleague’s teaching?  If you are not yet tenured (and some department chairs find themselves in that position), would you be able to critically judge the teaching effectiveness of colleagues who will be expected to write letters for your tenure application?  Imagine further that chairs have little or no training.  Why should we expect them to put themselves out? 

Should course evaluation be summative or formative?  Summative evaluation should be able to determine if an instructor employs effective teaching practices or not, using a holistic approach.  This is what good chairs do. But summative evaluation is likely to never be more than a rough judgment, suggesting that fine gradations are problematic. By contrast, formative feedback can improve teaching practice. Imagine a scenario where experienced teachers are invited to review one’s teaching.  These reviewers would provide “peer commentary,” which could be better received than “assessment.”  Teachers could choose to include such commentary in their annual review documents, but they would not be required to. Imagine if peer commentary was perceived by the faculty member analogous to the review of a journal submission.  The faculty member should recognize and accept that other, more experienced colleagues may have useful insights that are worth taking seriously.  The idea would be for the teacher to take what they think is valuable to incorporate into their teaching.  To effectively get “credit,” say for annual evaluation, tenure or promotion, the teacher would have to report the commentary as well as their response to it. Of course, this would require buy-in from faculty, chairs, P&T committees and deans to more carefully evaluate teaching effectiveness.

College teachers are conservative about pedagogy.  They need compelling reasons to make changes to the way they teach. A long time colleague asserted that he was an effective teacher and until he saw evidence to the contrary, he saw no reason to change.  He was a good teacher, but he could have been even better.

Absent training in graduate school about pedagogy, faculty tend to teach the way they were taught, which often means lecture.  There is a place for lecture, but to reach most students, lecture shouldn’t be the only pedagogy or the most dominant one.

According to a recent Chronicle article, “When STEM courses are taught using active-learning techniques rather than a standard lecture, students perform better, according to a widely cited meta-analysis … published in the Proceedings of the National Academy of Sciences journal.” In addition, “[W]hen much of class time is devoted to active learning, it reduces gaps in student performance.”   

Changing one’s teaching from predominantly lecture to something more active is usually perceived as a very heavy lift.  For experienced faculty, who have taught a variety of courses, changing one’s pedagogy in all their courses can be an immense challenge.  Where will they find the time and energy given their already heavy workload?  The answer is similar to the joke about how one eats an elephant.  The answer is one bite at a time.  

A colleague told me “Okay, I know that active learning techniques are more effective than lecture, but what would active learning look like in my [insert discipline] classroom?  I need specific ideas–I don’t really know where to start.” This is where most higher level discussions of improving teaching practice fall short. The best way to change one’s pedagogy is one step at a time. Start, for example, with an exit-ticket at the end of class. That doesn’t change anything other than the last few minutes of each class section.  See how that works and then add something else.

Conclusions

How might an institution’s strategic plan (and programs & policies stemming from that plan) be different if, instead of assuming that all faculty are good (enough) teachers, it was assumed that all faculty could improve their teaching effectiveness and that improvement would enhance student success in a measurable way?

The evidence seems clear that faculty will improve their teaching effectiveness when they adopt evidence-based instructional practices in their courses.  How can that be facilitated?

According to Cathy Davidson, the fundamental impediment to teaching improvement is lack of incentives for faculty to change.  Faculty earn credit for scholarly publications.  They earn almost nothing for improved teaching.  Publications can be counted and weighted.  Teaching innovations are not.  

Current teaching evaluation is designed to, at best, weed out the really poor teachers.  A better approach would be to help all teachers improve their practice.  How can this be accomplished?

Evaluation of teaching should be enhanced to change the focus from summative assessment to formative feedback. This can be done using a holistic approach as described above, where the goal is to identify good teaching practice and suggest ways the practice could be improved.

We need faculty development programs that are robust and provide time and perhaps (summer) stipends.  Like most things, teaching effectiveness depends on learning and experience.  Effective teachers are made, not born.  

The best faculty development is immersive. Faculty will change their practice more from participating in an immersive experience than from listening to a presentation. An immersive experience engages participants with genuine work that will be shared with others.  

Start with incoming faculty, making the program affordable and doable. Faculty new to teaching tend to be poorly prepared and would benefit from a significant teaching development opportunity, much more than is typically in place.  New faculty need more than a one hour session on how to use the institution’s learning management system.  They need a week of training during the summer before classes start, or better yet, a semester long-seminar on course design, bringing in good teachers to demonstrate effective practices.  That would also be a good way to build community.  Ideally, participants should be given a teaching reduction, but also be expected to produce deliverables: If the topic of one month’s meeting is active learning, for the following month participants should bring in one or more examples designed for their courses to be shared and discussed.  Requiring deliverables tends to result in much deeper learning, and in this case the products will enhance courses into the future.

Why not teach faculty to be more effective teachers? If we do not, it’s like leaving money on the table.  It’s an opportunity missed.

[ Answer to Inside Joke: It takes no free market economists to screw in a lightbulb, because if the lightbulb needed to be replaced, the market would have already done it. ]

A friend, Carrie Mook Bridgman, reached out on Facebook and asked my opinion of Rutger Bregman, “Why Society’s Biggest Freeloaders are at the Top” Here’s what I think as a mainstream economist.

This article is true in part, especially where it makes general statements. It is also misleading in parts, where it makes statements that are true in some cases, but not in all cases.

Is it workers who create products and wealth or is it owners of businesses? Can businesses operate without workers?  Generally no. Can workers operate productively without capital and entrepreneurship?  Generally no. The answer, then, is both workers and owners (i.e. capitalists) create wealth.  There are two fundamental questions here:

1. Assuming we wish to increase the average standard of living, we must increase the production of goods and services, and thus wealth. How do we do that?  The market system (i.e. capitalism) has proven to be a very effective way to do that in history.  It was the market system that motivated the Industrial Revolution that raised the standard of living in the developed world above the subsistence level. Is the market system perfect? Our market system in the 21st Century U.S. is not.
2. How do we (as society) divide up income and wealth?  The goods and services a society produces generate the income earned and the wealth created.  The U.S. has largely adopted the market system of economic organization which implies a certain way of dividing up income and wealth.  Let’s explore this in more detail.

The author of the article above, Rutger Bregman, observes that there are two ways of making money: earning it through work, and unearned income, which means income from savings.  The author implies that the former creates wealth, while the latter just expropriates it, or to put it in his words, “someone who uses their control over something that already exists in order to increase their own wealth… at others expense.” 

In order to create income and wealth, workers and businesses must create something consumers value. Workers working alone without the equipment and technology provided by the business aren’t very effective, which is to say they find it difficult to create much value.

Slight Wonking Out tangent: In a sense, the author is conflating two similar concepts: economic rents and rentiers. An economic rent is the difference between the return on an asset in a particular use and the return on the next best use of the asset.  Think about what Michael Jordan earned as a basketball player vs. what he would have earned as a golfer (say).  That difference is the unique return to his basketball skills.  Is it fair that Jordan earned so much playing basketball? Fairness is subjective, but we can say that his earnings were based on market forces.  As long as the demand for his services was as high as it was, he could earn as much as he did.  Because his skills were unique, he was able to capture most of the value from his skills. For most workers, this is not the case.

Compare this to a rentier.  In the 19th Century, a person who owned a lot of land, could rent the land out and live on the earnings.  Is that fair?  Again, that’s subjective, but we should consider a few things.  Where did the land come from? Suppose the person saved for years to purchase the property.  Then the rents he or she earned were based on their hard work and willingness to save. (Do you consider yourself a leach on society because you have money in the bank?  Probably not, but it’s the same argument with rentiers.)  But what if you inherit the land?  Does that change things?  Perhaps we could argue that it does, since  you didn’t work and save yourself.  But we also generally think it’s appropriate to leave inheritances to our children.  So the answer here isn’t completely obvious. End of tangent.

Businesses are successful when they create and sell products consumers value.  We don’t usually criticize grocery stores for selling groceries, because they provide a necessary service.  (Also, for what it’s worth, the profit margin on grocery sales is pennies on the dollar, so they make their money on volume, which requires happy and thus repeat customers.)

Businesses have strong incentives to create value, both with existing products and by creating new ones.  This is a benefit to society, both because the products have value and because the revenues generated are distributed to workers and owners.  

But businesses also face a conflicting tendency–to exploit market power.  Few people believe that an iPhone isn’t a marvelous product.  (Value creation)  But how does Apple manage to charge $1000 or more for one?  The answer is Apple’s market power–the ability to raise their price and not lose all their customers.  Just as Michael Jordan had market power, Apple is able to charge a high price for its product. In a sense, the problem is us, consumers.  As long as we are willing and able to pay high prices for products that we value, we will have to do so.  What is the solution?  The answer is competition.

Conservatives argue for free market capitalism.  They have valid reasons for this.  Free markets generate the most efficient economic outcomes, which means the highest average standard of living in a country.  Free markets exist when there is sufficient competition among firms that no firm has significant market power.  If Chevrolet charges much more for a pickup truck than Ford does, some people will buy Ford pickups instead.  That potential loss of customers keeps Chevrolet in line. As a result, firms earn only reasonable profits.

Liberals observe that in the real world many markets lack sufficient competition, so that the few firms that exist in a market have significant market power, allowing them to charge higher prices and earn high profits. A good example is pharmaceuticals.

How do we correct for businesses exploiting market power? Only the government has the power to be a counterweight, since government defines and enforces the rules of the game, i.e. how markets are allowed to work.  Does government always get it right?  No.  Governments are made up of fallible human beings, but absent government regulation, private markets will fail.

But wait, there’s more.  In an economy where government plays an active role in managing markets, there is the possibility of crony capitalism, where actors with power and money (e.g. corporations and rich individuals) can influence government officials to favor their interests.  This has also been described as welfare for capitalists.  The poor have often been criticized as takers (i.e. recipients of welfare).  But who is in a better position to obtain benefits from the government (e.g. thru the legislative process), the poor or the rich?  But in the case of the latter, we call it agricultural price supports, or defense spending. Economists are taught that if it quacks like a duck, it’s probably a duck.

Bregman also states, “[M]uch of the financial sector has become downright parasitic. How instead of creating wealth, they gobble it up whole.”  This is a bit of an overstatement.  During the lead-up to the Global Financial Crisis in 2008, yes, this was true; but in general, much less so. Banks provide mortgages to middle class Americans, and they provide loans to small and medium size businesses. Our lives would be very different without the financial system.

Bregman concludes, “Yet it doesn’t have to be this way. Tollgates can be torn down, financial products can be banned, tax havens dismantled, lobbies tamed, and patents rejected. Higher taxes on the ultra-rich can make rentierism less attractive, precisely because society’s biggest freeloaders are at the very top of the pyramid. And we can more fairly distribute our earnings….” On that last point, I agree.

President Biden recently announced an infrastructure bill of some $2 trillion. Infrastructure provides the underlying structure of a working economy.  Infrastructure consists of expenditures that have long returns benefiting a large number of citizens and businesses, but which tend to have high initial costs making it unlikely for individuals to be willing to “purchase” it themselves.

Infrastructure is non-partisan.  Catherine Rampell observes,

Survey after survey has found that investing more government money in “infrastructure” has broad, bipartisan support: 85 percent of voters overall, and 82 percent of Republicans specifically, agree that “America is in need of an infrastructure improvement,” a recent Morning Consult poll found.

Conservatives recognize (in principle) that infrastructure should be largely provided by government.  Who provides for the construction of roads, highways, bridges, airports, and most hospitals?  It is government at the local, state and federal level.  Why does government do that?  Because we recognize that infrastructure suffers from the free-rider problem—Once the infrastructure is there, everyone can take advantage of it, so no one wants to pay for it, at least not directly.  Sure, there are examples of private infrastructure—toll roads and privately-owned hospitals, but they remain the exception rather than the rule.

As commentator David Von Drehle stated,

Infrastructure is the connective tissue for human cooperation and communication. Bridges, roads, seaports, airports, railways, waterways, power lines, broadband — they all end up paying for themselves many times over because they leverage the greatest economic resource on earth: people.

The table below shows that the U.S. has spent less over time (as a share of GDP) on infrastructure, since the construction of the Interstate Highway System in late 1950s-1960s.  Significantly less.  During times of tight budgets—and when it is not a time of tight budgets—it is easy to defer maintenance on existing infrastructure.  After all, it’s not like a road is going to fall apart after a year.  And because new infrastructure is expensive, infrastructure investment is costly to finance.

Infrastructure investment is like maintenance on your house.  For the technically-minded, yes, I know that new infrastructure is different in concept from infrastructure maintenance, but they both have the same impact. You know you need a new roof on your house, but roofs are expensive and so you let it go for a few years.  Eventually, a big storm comes a long and you discover leaks as rain comes through your ceiling.  What do you do?  What you don’t do is say “I don’t need a new roof,” or “I want someone else to pay for my new roof.” Rather, if it is at all possible, you get a new roof, either using savings or through borrowing.  Borrowing makes sense since the new roof will last a long time so it makes sense to spread out paying for it. 

Even though infrastructure is a public good, when infrastructure reaches a certain point of depreciation, it exacts explicit costs on private individuals—think tire blowouts from pot holes and people unable to get healthcare when a hospital hasn’t been maintained and expanded to keep up with population growth. 

Infrastructure is, or should be, a bipartisan, no-brainer.  It should have been an easy win for the Trump administration, but as far as I am aware, they never managed to bring an infrastructure bill to Congress.  The fact that the Democrats are now in charge doesn’t negate the need for infrastructure investment—each year without it, makes the need bigger.

Upon the announcement of President Biden’s infrastructure bill, various Republican members of Congress indicated that they were opposed to the bill.  Let’s think about what could be behind their opposition:

• Concern about a large budget deficit.  That horse is long out of the barn.  Republicans lost their credibility concerning budget deficits when they passed the 2017 Tax Cut bill. This was an interesting experiment to see if a significant tax cut while the economy was at full employment could stimulate the economy.  The result was clearly no; instead, the result was a significant transfer of income from tax payers to the rich.  Sure, there were tax cuts for nearly every income group, but the vast majority when to the rich.

• Concern about what a budget deficit would pay for.  Republicans are in favor of tax cuts to the rich, and they are generally in favor of defense spending.  They claim that the proposed infrastructure bill would lead to socialism.  Yet, likely every member of Congress can find needed infrastructure investments in their districts. Indeed, Rob Whitman, the local Congressman for where my university is located, has argued for years for extension of broadband internet to all his constituents in rural, underserved areas.  Certainly, the composition of the infrastructure bill matters.  A good summary of the initial proposal is here.

From what is known about the Biden bill, there are a range of projects from more traditional infrastructure (roads & bridges, airports) to less traditional infrastructure projects, like universal broadband, to healthcare/assistance to the elderly, which is the least like traditional infrastructure.  If you don’t believe tax payers, especially wealthy tax payers, should pay for eldercare, you should make that argument. All of these projects are worthy of discussion, but for the entire package to be dismissed without debate as “socialism” is posturing at best. 

• Concern that such a large spending package, on top of the Pandemic stimuli, would cause significant inflation, something the U.S. has not seen in decades.  Traditional economic theory suggests this would be the case, but the world is changing in the Twenty-first Century.  It is possible that the infrastructure bill could be inflationary, but the Federal Reserve seems ready to nip any inflation in the bud.  More importantly, this concern conflates two types of spending: consumption expenditure (a good example of which is the pandemic stimulus checks sent out by both the Trump and Biden administrations) and investment expenditure.  Infrastructure is the latter, and will yield returns well into the future, serving at least to some extent as an offset to inflationary pressures.

• Concern about how the infrastructure bill would be paid for.  It is human-nature to wish to receive benefits that some other party has to pay for—see Free Riding, above.  The Biden administration claims to want to pay for infrastructure by partially rolling back the 2017 tax cut bill, which would mean that the rich pay the lion’s share.  The Republican’s natural constituency is the rich, so it is understandable that they would wish to avoid this.  Is it too much to ask that Republicans who believe this be transparent about it?  I guess it is.  The remainder of the infrastructure is to be paid for by borrowing—expanding the budget deficit.  This actually makes a great deal of sense.  Few people purchase a house with cash.  Since the house will last for decades, it makes sense to finance it with a mortgage; that is, by borrowing.  Furthermore, given how low interest rates are for the U.S. Treasury, borrowing money for infrastructure investment should be a no-brainer.  People are willing to lend to the Treasury nearly for free.  That makes infrastructure investment even more profitable and worth doing.  Okay, Republicans: If you don’t like how President Biden plans to pay for the infrastructure bill, how would you pay for it?

• The last reason Republicans might oppose the infrastructure bill is because they don’t want a Democratic administration to get credit for it—they don’t won’t government to be seen as doing something effective.  This would be unethical and dishonest, in my opinion.  I am not naïve enough to think that my opinion would prevent an elected official from doing the wrong thing because it is to their narrow political advantage.  It will be interesting to see if Congressman Whitman, after claiming for years to desire broadband for his unserved constituents, votes against the infrastructure bill.  I hope some Republicans will choose to vote for the common good.

Traditional publishers make their profits by exploiting monopoly power.  The traditional textbook industry is dominated by a small number of large firms, think Cengage, McGraw-Hill, Pearson.  Economists describe this type of industry as an oligopoly, in which the large firms have a significant amount of monopoly power, which has enabled them to increase prices significantly over time.

It’s not that the publishers are evil people; merely, that profit is their primary goal, and that goal drives them to price their products as they do.  How do they determine prices?

Traditional publishers are masters of price discrimination, which is a strategy to charge different prices to different customers in an attempt to get each customer to pay the most they are willing or able to obtain necessary course materials.  Think about how retailers offer goods for sale at regular price, and then put them on sale for the value customers who aren’t willing to pay full price.  Or think about movie matinees, which charge lower prices during the day when retirees and students can go, but working adults are typically working.

Publishers, of course, prefer students to pay for full price texts.  Unfortunately for them, prices have finally reached a level where students are increasingly unwilling or unable to continue to do so, especially in light of competition from open educational resources (OER), textbooks (and other course materials) that are free to use and offered by publishers like OpenStax and others.

Because of this competition, traditional publishers have rolled out lower priced models, like Cengage Unlimited, where students pay a single fee (currently $119.95 per semester) for access to all the Cengage books their professors adopt.  The more of a student’s courses use Cengage texts, the better deal this would be.  Faculty don’t generally choose texts because instructors in other courses have adopted their texts from the same publisher, so it’s not clear how good a deal this is in practice, though it is cheaper than paying full price.

Finally, if students won’t go for Cengage Unlimited, there is also Inclusive Access (IA).  IA is program by which students in a course get access to an electronic version of the text by virtue of registering for the course.  Universities negotiate for a discounted price, which publishers are willing to offer since all students in the course are required to pay for the book, and the university agrees to bill the students directly.  In other words, IA puts student on the hook with their university to pay for the course materials, whether or not they choose to use them.  Survey data indicate that many students, if not most, go without the text for expensive classes.  Traditional publishers are willing to offer discounts through IA, since 100% of a discounted price is preferable to the publisher than 50% (or less) of full price.  In other words, IA is best described as text rental w/automatic billing by the university. Students do have the right to “opt out” of obtaining the text through Inclusive Access, but this option is not well publicized.  Course materials obtained through IA can cost less than $100 depending on how large the university/system is and how well each university/system negotiates the price.

Publishers of OER have rightfully publicized the significant money students have saved when faculty adopt OER.  The commercial publishers have recently coopted this marketing device with images like the one above.  But there is a significant difference.  The money OER saves students is money that comes out of traditional publishers’ profits.  By contrast, the money traditional publishers “save” students, when they offer lower price options is money they previously took from students.

Should we give credit to traditional publishers for offering lower price options?  Sure, but don’t imagine they are doing this out of the goodness of their hearts.  Given the competition from open, traditional publishers are attempting to charge the most they can get away with.  In other words, the lower priced options are the best traditional publishers can do.

Don’t be fooled!  I’ve never seen a traditional publisher offer a better price than OER.

In recent years, most colleges and universities, at least in the U.S., have adopted explicit initiatives to promote student success, which they assess aggregate metrics in terms of retention and graduation rates.  Yet at the same time, I wonder if there aren’t institutional structures that get in the way of individual student’s success, often times the individual students who need help the most.

Think about the transfer student coming from a community college to complete his last two years at a four-year school.  He transfers in during the summer, after the normal registration for fall courses is complete.  He has all of his general education courses done, and many of his electives.  He finds it difficult to find open courses in his major.  He ends up taking a minimal full-time course load with courses he hopes will count towards graduation.  He can only get into his major during the spring semester, which means he is unable to graduate by the end of his 4^th year in college.  It could be worse.  Suppose he is in a major which is vertically building, meaning the major requires a sequence of courses to be taken in order.  In this case, he misses a full year in the sequence which requires an additional year to graduate.

Consider a student who decides to change her major at the beginning of her third year.  Last Spring, she registered for a full course load, but those were courses in the previous major.  With the new major, she requires an entirely different set of courses, most of which she can’t get into as she runs into the same problems as our transfer students above.  Maybe she feels like she’s wasted the semester.  Maybe she feels like dropping out.  Maybe she can’t afford a fourth or fifth year of college.

This is not a picture of student success.  Can universities do something about it?  As long as the school’s aggregate metrics are improving, maybe they don’t perceive this as a problem.  But, in my view at least, students are not an aggregate.  I don’t teach courses, I teach individuals.  How could we change these outcomes?

Faculty advisors are in the best position to see these problems, if they are paying attention.  But faculty advisors have limited power.  If one of my majors needs to get into a closed course in my department, I will go talk my colleague, the instructor.  I’ve even done this occasionally with colleagues outside the department, though not when I don’t know them.  Transfer students and new majors are more problematic since advisors don’t yet have a relationship with them.  Perhaps it’s the department chair in the new major who is in a position to find them classes in the department, assuming the chair is aware of the problem, and willing to make the effort.  All it takes is one faculty or staff member to step up and carry the load, to help the student get over this hurdle.  But often times this feels like extra work.  I wonder if students don’t need an ombudsman, someone who is tasked with helping in situations like this.  Ideally, this would be someone in the new major.  Otherwise, it may need to be someone with power, like a dean, to force the situation.  How many students with these kinds of problems are we willing to let down before we fix this problem?  Ten per year?  Twenty-five? How many?

Image “Ombudsman Scrabble” courtesy of House Buy Fast via Flickr.com

My intro students always seem to do less well than I expect on exams, especially on the final exam.  Part of the reason is that I ask at least a few questions which go beyond what we’ve explicitly done in class.  I do this because I think exams should stretch the students: they should be a learning experience as well as a summative assessment.  I don’t want to simply ask students to memorize and regurgitate on the exams.  I stand by this, but now wonder if this approach needs an adjustment.

Assessing final exam performance is complicated.  My final exams are cumulative, in part because economics is cumulative.  Roughly half the final exam assesses content we’ve done since the previous exam (I give two midterm exams), so it’s content students haven’t been tested on before.  In addition, I’ve tended to have to rush things at the end of the semester, so students haven’t had enough time to process the most challenging content in the course.  One last point: Students need to study for other final exams at the same time at the same time as they prepare for mine.

You may recall that I use Lumen Learning’s Waymaker as courseware in my intro courses. One manifestation of the problem here is that students can do very well on the Waymaker module quizzes, but still get significant numbers of my exam questions wrong.

In my last post, about the CTREE conference, I wrote about Stephen Chew’s keynote address.  Chew observed, “Many faculty take pride in how hard they make students struggle.”  Such faculty seem to think that the more students struggle, the more they will learn.  I believe in rigor, but rigor is in large part about setting expectations for the work.  The struggle hypothesis is different. if you think about it, it cannot not be entirely true.  If you give students an advanced lecture in an intro course, they won’t get it no matter how hard they struggle. If you give students an exam from a downstream course, they are unlikely to be successful. More importantly, the struggle hypothesis is not supported in the learning science literature.  We want students to engage deeply with the course content. How can we reconcile these views?  Chew does this by using the concept of desirable difficulty.

We want students to take their studies seriously.  We want them to think, even struggle.  But the work has to be within their capabilities.  We want them to stretch, but it can’t be too far.  As my friend and colleague, Jeff McClurken says, students should be “uncomfortable, but not paralyzed in their learning.”

This reminds me of Vygotsky’s zone of proximal development. The zone of proximal development refers to the difference between what a learner can do without help and what he or she can achieve with guidance and encouragement from a skilled partner.  I need to do a better job of providing that guidance and encouragement in my intro courses.

Given Chew’s comments on “desirable difficulty,” I think I need to revisit what David Wiley suggested to me years ago about the gap between Waymaker module quizzes and my exams—I need to think about reducing the gap.  More precisely, if the exams assess what I want students ideally to learn, then I need to better prepare students for what the exams are assessing.  This means rethinking the content and learning activities.  I try to do this in my lectures, but the last week of school tends to be rushed.

This isn’t a novel idea for me: I went through this process some years ago in my intermediate macro course.  I had a clear idea of what I wanted my students to be able to do by the end of the course.  So, I intentionally built instruction around that goal, starting from the first day, through regular homework assignments using different content but the same process.  How can I do this in my introductory courses?

I will start by developing a/revisiting my list of learning outcomes. (Some years ago, I created an outline of the content in my course.  It wasn’t explicitly a list of learning objectives, but it included many.)  Once I have that list, I plan to share it with my students as part of the course outline.

Next, I will review/revise the course content (text, lectures and learning activities) to be in alignment with the learning outcomes. Then I will review/revise the module quizzes and make sure all is in alignment with the final exam questions.

I will do each of these tasks starting at the end and working forward, so I insure adequate attention with the content at the end of the course as well as at the beginning. This is my task for the summer. In the fall, I will assess how well the process has worked.

If you think this sounds like a lot of work, I am certain I can do it in the time available this summer.  If you wonder why I am willing to spend my summer on this project, I will tell you what a visionary colleague once told me in a different context: If you think this is important to your students’ learning, how can you not invest the time?

Last month I attended the 2019 Conference on Teaching & Research in Economic Education (CTREE), sponsored by the American Economic Association, in St. Louis, Missouri.  The sessions were very high quality and I learned a great deal.  One thing that struck me was how little of what I learned was specific to economic education.  Hence, the reason for this post.

Two important themes came out of the sessions I attended.

• How little we actually know about teaching and learning. Learning is complex, more than most realize.
• The critical importance of student effort, something which researchers often ignore because of the difficulty in measuring it.

The opening keynote speaker was Stephen Chew, a prominent cognitive psychologist, who readers of this blog will know I’ve followed for some time.  Chew’s premise was that students and teachers each have a model of learning in their heads, which guides their actions.  For some teachers, the model is implicit.  A teacher’s mental model determines which teaching methods they select, how those methods are implemented and assessed, and how to make adjustments.  The problem is that most of these mental models of both teachers and students are flawed, which makes teaching and learning less effective.  In this post I will focus on how to teach more effectively.  Chew has created a set of videos to help students learn, which can be found here: http://www.samford.edu/how-to-study/.

For me, the most interesting part of the keynote was Chew’s discussion of cognitive load.  This may be because in my earlier studies, I didn’t fully grasp it.  Anyway, let me give you my current understanding.  In order to complete mental tasks, a person applies mental effort.  Mental effort is always limited.  Cognitive load is the total amount of mental effort a task requires to complete it.  A person can do multiple tasks as long as the total cognitive load is no greater than available mental effort.

Memory is divided into working memory, where new ideas (e.g. data) are placed, and long-term memory, where knowledge and understanding reside.  Learning is stored in long term memory in the form of frameworks or models. These are mental structures that give meaning to data.  Cognitive psychologists call these “schemas.”

Things in working memory need to be processed into appropriate schemas in order to be moved into long term memory.  But working memory is limited—Chew says it can hold only about five “chunks” of information (though experts can hold more).  If you try to put more into working memory, the earliest contents get flushed before making it into long term memory.

When I first learned about working and long-term memory, I kind of discounted the theory because it sounded like memorization of facts.  Learning is more than memorization of facts and I now see that that is implicit in this theory.  We know that novices learn less efficiently than experts do. If teachers don’t recognize this, we may end up trying to do too much, with the results that students don’t learn what we’re teaching. This is related to the “curse of expertise,” which Chew explains as “the more one knows about a topic, the harder it becomes to remember not knowing a topic and the effort required to learn that topic.”

If you’ve taught for any length of time, you will have experienced those classes where at a certain point, a majority of your students are staring blankly at you.  If you’re paying attention, you decide to end class early.  I think this is where the cognitive load exceeds the mental effort available.  If you continue the class session, you may “cover” more material but the students won’t get it.

Novices perceive new data as random facts—they have no schema to frame the data.  Experts have existing schemas, so when they perceive new data, they file the data in the appropriate framework.  That’s usable knowledge.  Data needs to be linked to existing knowledge to make it usable (i.e. part of the schema).

There are three types of cognitive load:

• Intrinsic: How difficult is the subject?
• Germane: Load caused by pedagogy and activities to learn the subject.
• Extraneous: Additional load caused by factors unrelated to learning the subject (e.g. random animations on an instructor’s PowerPoint presentation).

If the cognitive load demanded exceeds the student’s available mental effort, then learning will not occur.  There are two issues here.  First, is understanding the subject being taught (i.e. the data).  Second, is fitting the new data into a schema.  The takeaway here is that teachers must monitor, manage and minimize cognitive load to allow schema development; they should also design activities to promote schema development.

As teachers, we need to eliminate extraneous load, and think about how to minimize germane load.  Note that there appears to be a tradeoff between intrinsic and germane loads.  The greater the intrinsic load, the less cognitive space is left for germane load, suggesting simpler pedagogy.  This is something to think about.

The crux of learning is the processing of new data into an existing schema.  This processing requires mental effort.  It’s often said that students learn best when they can relate new information to something they already know.  If they have no existing schema, this processing is harder, which explains why novices learn less efficiently than experts.   Since many undergraduates have little or no experience with economics, this adds an additional challenge to an introductory course.  Let’s call “study” the mechanism of processing new data.  Learning scientists have determined that some study practices are more efficient than others.  Reading and highlighting text, while popular among students, are known to be among the least effective ways of studying.  Deeper learning requires building lots of links between new data and existing schemas. Reading provides only a weak link.

Herb Simon observed, “Learning results from what the student does and thinks and only from what the student does and thinks.” He continued, “The teacher can advance learning only by influencing what the student does to learn.” Koedinger et al (2016, p.28) found “[t]he learning effect of doing is about six times greater than that of reading.”  Yes, that’s correct.  Doing (i.e. working with the content) yields six times the learning of reading.

What does this mean for course design?  What is doing?  Doing means working with the content. Working with the content builds more and stronger links with one’s schema.  Skimming a chapter is pretty much the opposite of this.  Instead, as one reads, one needs to think about what one is reading.  How does what one is reading about relate to what one already knows? Chew calls this Elaboration. Everyone knows (or thinks they know) something about economics.  Chapter 2 can be related back to Chapter 1.  How is the content different from other concepts? For example, how does accounting profit differ from economic profit (Distinctiveness)? How is this relevant to my life (Personal)? Can I think of any examples?  Finally, how might I be expected to use this content to demonstrate my understanding (Appropriate to Retrieval and Application)?

Cognitive psychologists have identified the testing effect, also known as retrieval practice with prompt feedback.  From what I understand, students who test themselves, even if they get the problems wrong, will be more likely to get them right on exams.  The point is not to score well on your self-test, it’s to build connections between the concept and your schema.  In short, assessment is integral to the learning process.  This benefit goes away if you look up the answers.  You need to force your brain to think about the question, even if you end up getting it wrong.

Two other things which promote deeper learning are spaced practice and interleaving.  As I understand them, spaced practice means spreading out your study.  Instead of (hopefully) reading the text and then reviewing the material just before an exam, it is preferable to spread the review out over multiple days or weeks.  Interleaving means blending older concepts into the study of new material.  This forces students to discriminate in retrieving their knowledge (e.g. how do I answer this kind of question?) and leads to deeper connections with the schema.  One thing I plan to try in my intro courses this fall is to give students a short assignment each Friday, based on the content we learned the previous week.  So after studying new material during the week, I’ll ask them about previous material.  This seems like a low-cost activity that’s worth a try.

I am not done thinking about what I learned at the conference. Stay tuned.

I was looking at the gradebooks for my last several semesters of intro courses, both face-to-face and online, and was reminded that there appear to be significantly higher rates of students failing to complete my online sections, either when students drop the course (after the first week) or when students fail to earn enough points to pass.  This is fairly widely known, but I began to wonder if it has less to do with the pedagogy and more to do with the student demographics.  More precisely, I wonder if the same factors that make online courses attractive lead to, or at least contribute to, those higher failure rates.

Adult students, for example, have busy lives with families and jobs to be responsible for.  They may also have less time, flexibility and/or ability to react to shocks.  A sick family member imposes different stresses on a parent who is a college student than on a traditional age undergraduate.

So perhaps we should expect lower success rates for online courses, rather than thinking of it as a shortcoming of them.