Monthly Archive for January, 2008

Response to Gardner’s Rock/Soul/Progressive: II

Published on January 11, 2008 in First Year Seminar and UMW Teaching Center. 1 Comment

I started to leave a comment on Gardner’s second post about his
recently concluded First Year Seminar, but found it was turning out to be longer than a comment typically is, so I moved it here.

After your previous post I was in awe that you appeared to have found the magic bullet for teaching an FSEM without the snags the rest of us faced. I’m glad to see that you ran into many of the same that I have experienced. ;-) I wish there was a way that the FSEM instructors as a group could collaborate on developing a list of such snags. I think a collective approach to addressing them would be more efficient and probably effective than each of us working on our own. Also, it might be helpful for new FSEM instructors to have a better idea of what to expect, and how not to panic when these snags occur. (Is there perhaps a role here for UMW’s upcoming Center for Teaching Excellence?) Let me respond to some of the specific points you made.

The day I asked the seminarians why they weren’t more lively in the question-and-answer period that followed their classmates’ presentations, and they replied that they didn’t want to ask questions for fear of making their peers look ignorant or stupid. … for most of them it was the absolute truth: they didn’t look at the Q&A as a time to go deeper with what their classmates had already showed they knew, or to bring in interesting connections, or generally to take the level of engagement and enthusiasm and inspiration up a notch or ten.

I ran into this as well. I’ve blogged before about how this year I’ve become more aware of how first year students are different from upper class students, and how they need explicit guidance about how higher education is different from high school. I expect Gardner’s observation above is common with first year students, and it’s something we should put on our common list of FSEM goals: to teach students this deeper purpose of Q&A. It’s not something that comes to them naturally.

[T]he way so many of the third presentations suddenly gelled into the kind of deep, thoughtful, rigorous, playful work I’d been hoping for–and trying to encourage–all along. …

There were times I thought we wouldn’t get there.

I saw this as well. Just when I was starting to give up hope, my students showed the depth of work I was looking for.It seems to take first year students longer to get what we expect in university-level work than upper level students. This isn’t surprising given their training and experience. So an entry level college course need to be less rigorous in content than an upper level course, but also different in terms of instructor’s expectations for how students will make progress towards learning how to do university-level work.

Then there was the day when it became clear that I’d have to tell them they should blog twice a week [emphasis added], when I had hoped that with this small group and a topic of some urgency to all of us music lovers, I could just step back and let the blogging commence.

This is another manifestation of the previous point. First year students need to be more explicitly told what the expectations are. It’s not that we dumb down the type of analysis and other work that we expect of first years, but rather that we need to provide more structure, more scaffolding about how to achieve the type of work.

After two tries at teaching an FSEM, I think that one of the key issues in achieving success is arriving at the right balance between what Gardner calls structure and emergence. The mistake that I’ve made twice now is to put too much emphasis on the latter, but I am getting closer to the target.

At the risk of a bit of reduction, I think the distinction between task-orientation and inquiry-orientation that Gardner raises reflects confusion in the students’ minds about who’s in charge. Are they satisfying an instructor’s requirement by completing a task, or are they searching for the answer to a question they find gripping. It’s the latter that FSEMs are trying to promote, but it’s the former that most students (even upper level students) have experience with.

Another sign of success is the extent to which students adopt a practice of self-reflection about their learning. Obviously, instructors can promote this or not, but when they don’t it’s very unlikely to occur. I was pleased to see that Gardner, like me a year ago, experienced a “Shannon Moment” when an FSEM student demonstrated that they “got it”:

Perhaps the most startling moment of all came after the class was over, when a student in my freshman seminar commented on my Theme Parks and Sandboxes blog post. It was as if something I had been saying over and over, all semester long, had suddenly connected.

Of course, Gardner identifies the most important factor for success in an FSEM:

[M]y best chance is always to let my fascination with the subject carry everything else along with it.

One final observation about the FSEM. I’ve had more interactions down the road with my former FSEM students, than with my other former non-major students, so it appears that the FSEM enables a stronger connection than a general education course.

Open Source for Higher Education

Published on January 11, 2008 in Open Source. 0 Comments

Dean Dad recently posted on a topic many of us are interested in: the potential of universities replacing their proprietary systems with open source solutions. Might be relevant for anyone thinking about the Bluehost Experiment and it’s downstream implications. (Private joke!)

A YouTube like Venture for Intellectuals

Published on January 7, 2008 in Social Networking. 0 Comments

Don Coffin, a colleague at Indiana University Northwest, just brought my attention to this interesting article in the New York Times.

Chicken Little?

Published on January 7, 2008 in Economics and First Year Seminar. 2 Comments

This post originated in a series of conversations I had over the holiday with my in-laws who were children of the Great Depression. But it’s also something I’ve thought about for a long time. The catalyst for posting probably came from Shannon’s recent missive “Why Wait?”. If you’re looking for thoughts on pedagogy, this post probably isn’t for you.

Two years ago, I had an interest in globalization, but didn’t really know that much about it. Having twice taught a first year seminar on the subject, I have turned an interest into at least a modest amount of expertise.

I generally take an optimistic view of life. I believe in the American Dream. As a mainstream economist, I also believe that international trade and investment are beneficial for both countries participating in such transactions. Though some individuals in both may be harmed, the gains to each country in the aggregate offset the losses. Change always has losers as well as winners, even change that is ultimately for the good. What I’m beginning to question is how long the US will continue to be among the most economically affluent nations in the world. What I suspect may be coming is that the US will continue to grow, but certain other countries will grow more quickly and surpass us. China comes to mind. This outcome is not a certainty, but what’s clear is that the US has no manifest destiny to remain at the top of the economic heap. What will be required to keep us there is innovation, investment, education, and hard work. I’m beginning to wonder if we’ll be able to maintain those. Why? Because I think most Americans, especially the younger generation, have become complacent about their economic affluence.

As an educator, I spend a great deal of time interacting with students. As a parent, I do the same with children, my own and many others. I’m also a child of the 1960s, and had parents who lived through the Great Depression. At the risk of being reductive, here’s what I’ve concluded from these interactions:

* My parents’ generation believed that if they didn’t work as hard as they could, learn as much as they could, and save as much as they could, their families would go hungry (no hyperbole intended). As a result, they achieved a significant of economic affluence as evidenced by the growth of the middle class in second half of the twentieth century.

* My generation believes that if we work hard, learn much, and save, we will be economically successful. And we largely have been, as illustrated by the wealth of the baby boomers.

* The younger generation seems to believe that they will be economically successful, whether or not they work hard, learn or save. And as a consequence, they don’t seem to be doing those critical activities very much.

My students don’t seem to work very hard at school. They don’t seem to take college that seriously, and they don’t see learning as their full time job. Most of them seem to have part time jobs, not to pay for school-related expenses–that I could understand–but rather to pay for what to me at least are luxuries. Some students have told me that they take part time jobs because they have free time! I often wonder what these students think they are in university for? Shannon makes the same point, I think, when she notes

College is a unique environment unlike anything else we will ever experience, yet we focus on other things, even ignoring the reason colleges where created in the first place.

At one level, these students seem to be behaving the way I remember kids in high school doing: kids who were not particularly engaged in school taking a part-time job where the money was “good” and then making a career of it. I fear that many of my students will being doing essentially the same thing: not staying in a part time job, but ending up in second tier jobs because they didn’t work to their potential in school. And I worry that more and more jobs like that will be the norm in this country because Americans aren’t willing to put in the work necessary to be qualified for something better.

Some of this is to be expected, what economists call an income effect: As people become more affluent, they work less and take more leisure. The problem I see is that as a nation, we seem to be behaving myopically. If we did this wittingly that would be one thing, but I wonder if most Americans understand what’s going on? And even if we do, don’t we have a responsibility towards future generations?

There are places in the world where people are hungry, figuratively and literally. Increasingly they are competing with us for jobs. Increasingly I think they will be winning them. Americans will end up taking second-tier jobs with second tier incomes. This will happen slowly, and we won’t even notice until it’s a fait accompli. I hope I’m wrong.

Teaching versus Grading or Learning versus Assessment

Published on January 1, 2008 in Teaching and Learning. 1 Comment

Some weeks ago, I mentioned the SCHEV course redesign conference I attended in November. A theme of the conference was how to use the power of computers to teach more efficiently and effectively. (This is the premise of Aplia, which probably does a better job than I witnessed at the course redesign conference.)

I went to the social sciences session at the conference, which was presented by a psychology instructor. He explained how he had redesigned his Introduction to Psychology course to fit the above mentioned theme. What was interesting to me was the central role that online quizzes played in his course, not merely for assessment, but for teaching and learning. He had constructed (or otherwise obtained) a large set of multiple choice questions, two or three thousand of them. For each course topic, students were assigned the online quizzes. Not only did the students take the quizzes, but they were encouraged to take them as many times as they liked, since only the highest grade was recorded. The instructor intimated that the students didn’t necessarily read the book (or even come to class); rather the quizzes were the course. I’m overstating here, but not by much. Students learned by doing the quizzes, rather than by reading the text or listening to lectures.

This concept is quite interesting and it certainly made me think, being a step beyond the idea I’ve explored of regular, low-stakes assessment. The problem that I’ve encountered with such assessment is that poor questions limit the usefulness of the assessment. That is why this past semester I told students that success in the online quizzes would assure them (only) a C+ in the course. In my experience, publishers don’t put near as much care and effort into test banks as they put into text books. (I certainly didn’t when my publisher asked me to add problems to my book.) While I use test bank questions, over the years I’ve modified them to better fit my assessment needs.

At the Course Redesign conference, as the psychology instructor demonstrated his quizzing software as well as some of the questions, I found myself seeing potential if not actual problems. What were the students actually learning? The quiz questions seemed to emphasize recall rather than any higher order skills. I asked about that and got two responses. First, since each time a student takes a quiz they get a different draw from the pool of questions, they’re not necessarily getting exactly the same questions, but rather similar questions on the same topics. But even if students weren’t getting the same questions, perhaps they were learning more about how to be successful on the quizzes, how to suss out the correct answer from the form of the questions, than what the correct answer was. The response to this was, “It’s only a 100-level course, so if they’re just memorizing concepts, I’m okay with that.”

This made me start thinking about the differences between learning and assessment, between how we teach students to learn, and how we test that learning. Returning to the previous context, if the assessment makes students go through the process of solving the problem, then using the assessment process to learn is fine. But if the assessment doesn’t require students to go thru the process of solving the problem, and I suspect most multiple choice assessments do not, then students can “learn” how to answer the questions without learning the underlying content. In this case, using assessment to “learn” is not fine.

Often, I think I test differently than I teach. I don’t think this problem is unique to me—it may reflect my disciplinary approach. The fact that students find economics (exams) “hard” may be a symptom of this. The basic problem is transfer, a form of external validity. We teach in one context, but ideally we want to see how students do in another. If we test on the same examples we use to teach, we can’t necessarily assess how deep the learning is, since we’re not assessing transfer ability. As such, the test is subject to the same criticisms I raised above about the psychology class. As teachers, what we want students to be able to do is use the same content correctly in a different context. That’s transfer. And that’s what I tend to ask for on exams. Not exclusively, of course. But the trick is to find the right balance between assessing basic content and transfer. (A further wrinkle is that I like to make exams a learning experience as well as an assessment. That is, I try to teach students something about the topic via the exam, too.)

My challenging student this past semester made me reflect more than I normally do about the way I assess student learning in my intro courses. After the second exam, he asserted that the appropriateness of my test questions should be based on their ability to measure achievement of the course goals. I agree. After reflecting, I decided I’m not sure the students and I are on the same page about what those goals are. Of course I have goals stated on the syllabus, but I’m not sure that my students have internalized them. Principles of Macroeconomics isn’t primarily an economics course—it is, or should be a general education course designed to introduce students to the social sciences. I think my course does a pretty good job of that. But I’m not sure the course exams have the balance right between the general education goals (which emphasize lower levels of cognition) and the economics goals (which emphasize higher levels). Or more precisely, I’m not sure the students correctly understand that balance. There are complicating factors here: Most of my students are first semester freshmen. The early material in the course focuses on the general education goals and is very easy to master. By the time students discover they’re not mastering the more complex material using the same (high school) study habits, it’s late in the semester and hard to change.

I think my efforts over the last few years to teach students metacognition were an attempt to fix this disconnect. By giving students more practice doing the kinds of problems they would encounter on exams, I hoped to teach them better as well. The problem that I ran into was that the online quizzes which I used for this purpose, coming from text book test banks, mostly assessed lower level learning. Thus, they weren’t teaching students all that I wanted them to learn.

I began to reflect on my intermediate macro course, where I think I do a better job of assessment. The students find the course difficult, and the tests challenging, but they’re not surprised by the types of questions as they seem to be in the intro course. The reason is that we practice those types of questions. That type of analysis is, in fact, exactly what economists do. My reflection made me wonder if I shouldn’t give students more practice of this sort. I’ve tended to limit the amount of practice because, more so than in the intro course, there is only a limited number of questions I can ask. Economic theory is fundamentally about using theories (or models) to make predictions about how events affect the economy. In the intermediate course, we only learn a handful of models, each of which has only a handful of parameters, so given that I want to ask students to use a model to assess the effects of changing one or more parameters, there aren’t really a large number of possibilities. I tell the students not to “try to memorize the different cases,” but to learn how to work thru whatever scenario is presented. If we work through all the options in practice, then how do I assess transfer?

But this raises another set of questions: Should every student be able to learn to do this? I want to say yes–if not, why not? Should every student be able to earn an A? What exactly should a grade mean? Are we confusing grading with teaching? We shouldn’t teach less well so that we can more easily discriminate between different levels of learning—Are we teaching students to analyze, which is what we say, or are we asking students to infer how to do it, something only the best students can do well? What does it even mean to say that a student is “one of the best?” In practice, it means they scored well on exams and other assessments. But does that mean they brought more to the course in terms of intellect or experience? Does it mean they learned more than most students in a value-added sense? Does it mean they learned to do economic analysis at a higher level than the one they’re in? Say at the level of a senior major for a freshman taking the intro course, or at the level of a graduate student for a senior major?

While I do a good job of balancing these issues in the intermediate course, I think I’ve erred on the side of protecting the sanctity of the grade rather than insuring that students learn. I’m going to make a concerted effort to change that this coming semester by giving more problem sets based on the types of analysis I want students to learn well enough to transfer. If I’m right, it should raise the mean exam scores (since more students are learning how to do macro well) while compressing the distribution of scores. Hopefully, I’ll still be able to discriminate between different levels of learning. It probably won’t affect the mean grade though since I’ve always ended up curving those.

This brings me back to the intro course. If the problem is getting students to practice more of the types of analysis I want them to learn, the solution is to come up with more good questions for them to practice on. That’s my next goal. One possibility would be to adopt the Aplia package, even though I don’t plan to use all the bells & whistles, merely the better quality quiz questions. Another possibility would be to apply for a course redesign grant to buy some time to create a larger personal testbank. I have only revised and expanded my test questions incrementally before, because devising good quality questions will take a great deal of time and it hasn’t seemed to be worth it or valued by the institution or the discipline. But perhaps it’s time anyway.

Spam prevention powered by Akismet