The Other Experiment

Last semester I conducted three experiments with my teaching, only two of which (teaching metacognition in my intro class, using blogs as a window into my students research process) I have blogged about before. The third experiment resulted from the coming together of three threads I’d been mulling over for a couple of years: how to get students to read the text, how to usefully incorporate formal critical thinking skills into my teaching, and how to use popular media in a non-trivial way in my classes.

What provided the critical mass was my discovery of a very thoughtful little book, The Guide to Critical Thinking in Economics. In only about a hundred pages, this book teaches a short course in formal critical thinking skills and argumentation, but from the perspective of economics. What I decided to do was adopt the book as an ancilliary text, along a traditional text, Sexton’s Exploring Economics, and the Wall Street Journal. Roughly twice a week I held normal lectures, but then Friday became our “lab” period in which we worked on critical thinking skills. Each week we covered a couple of sections in the Epstein & Kindleberger book and I gave students a few problems to test their learning. The problems were slightly modifed from the book, where students (in small groups) were asked to define concepts and then find examples to illustrate them from their textbook (Sexton) or the Wall Street Journal. My hope was that doing these assignments would encourage them to read the course texts.

One of my goals in this experiment was to teach students economic literacy, narrowly defined as the ability to understand the economic arguments in popular media. I thought that instruction in critical thinking skills could enhance students’ ability to make sense out of the articles in the Wall Street Journal. I reinforced this by having students write a series of three essays on articles I selected from the Journal based on their economics content. You can find links to these assignments below.

Each assignment had several parts, a preliminary part which had students decipher the article on which the assignment was based, and then one or more subsequent parts in which they were asked to write about the article in increasingly complex ways. The first part of each assignment asked students to identify the economic principles or theories appealed to in the article, and then to outline the main points in the author’s argument. The first essay asked students to write a prose summary of the argument. The second essay asked for a prose summary, and then asked students to evaluate the logic and evidence of the argument. Next, it asked whether they agreed with the argument, and what objective or subjective standards lead them to their conclusion. (In other words, they could find the logic sound, but disagree with a premise, or object on some normative grounds.) The third essay asked for a prose summary, and then asked students to change the key assumption in the argument and argue to a new conclusion. The results were quite interesting.

Last Fall, I drafted a paper on this experiment. The paper included statistical tests of two hypotheses: first, did instruction in formal critical thinking skills enhance economic lliteracy as measured by the quality of the essay assignments, and second, did the instruction also enhance learning of economics more generally as measured by the scores on the course exams? When the paper was done I realized I’d left out much of the narrative I had discovered through the experiment, since that narrative didn’t fit well in the frame of a scientific paper. So, the purpose of this posting (with lengthy preamble) is to blog about that narrative here.


* Some students had trouble differentiating between summarizing the argument and summarizing the article; they got side tracked discussing issues tangential to the argument. It was as if they were afraid to leave anything out of their summary–Is this a carryover from high school?

* Some students didn’t clearly differentiate between their opinions and those of the author.

* During the first semester I tried this experiment, students either didn’t do the first part of the assignment (identifying the economic principles or theories in the article) or they didn’t incorporate these principles into their summary of the author’s argument. More generally, students seem to have trouble seeing the economics, as if they don’t make the connection from the prose in the article to the underlying concepts. If I ask bluntly, “what is the economic principle embodied in this paragraph?” they can answer correctly. But if I just ask them to summarize the argument, the economics tends to be omitted. One fix I tried was to point out to students that newspaper articles are a condensed form of argument—that they include code words (efficiency & tax reform) that may require more research to decode. I encouraged them to look up those terms in the textbook.

* It is also easy to overestimate undergraduates’ knowledge of economics in the real world. In both the first and second year I tried this experiment, I chose a topic which I assumed students would understand, but they did not. This made the assignment less effective. The topic that fell into this problem the first year was health insurance. The topic from the second year was income tax. Because most students turned out to not understand how these work in practice, they fundamentally didn’t “get” the assignment.

* Some students had trouble reading beyond the rhetoric in a newspaper article. They didn’t see the objective argument since the rhetoric made them believe it was entirely subjective. The fact that an author is opinionated doesn’t make him wrong. It didn’t occur to me that students might need some instruction in critical reading, but in retrospect some did.

* Most students didn’t seem to put enough effort unearthing the author’s argument or critically thinking about it. They stopped with superficial analysis. For example, students often made criticisms that begged the question; e.g. “The author is a journalist not an economist.” Therefore, his economic argument doesn’t make any sense. “The author doesn’t support his logical argument with empirical evidence.” Therefore, his logical argument must be wrong. The students didn’t actually state these conclusions, but they implied them. Note that these types of criticisms are much easier to make than actually analyzing the argument as it exists. Similarly, some students heavily discounted predictions or simulations as “mere speculation,” not hard evidence, even when based on standard econometric techniques which economists would accept as reasonable.

At the same time, in these cases it wasn’t clear that students knew they were taking the easy way out. Rather, it seemed to me that they simply didn’t know or lacked experience in how to critically analyze an argument both for what it says and what it leaves unsaid.

* When asked to draw a conclusion about an argument, some students choose to not decide, claiming they don’t know enough about the issues. While this is something that the critical thinking text argued for, I wondered if it was also an example of multiplicity on the Nelson-Perry Scale of Cognitive Development, the belief that there is more than one view on a question and they can all be correct. This type of cognition is typical of first year students.

This entry was posted in Economics, Teaching and Learning. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *