Archive for the 'Assessment & Grading' Category

The most important job of a university teacher is designing the learning environment

Published on January 8, 2010 in Assessment & Grading, Teaching and Learning, The Future of Higher Education and What is Education?. 4 Comments Tags: .

Writeable WallWhat does a teacher at the university level do?  This is a question I’ve pondered for some time, as I’ve been following changes in higher education, for example, the growth in the for-profit sector, and the growth in on-line learning.  There seems to be a growing trend for instructors to be assigned to teach courses they didn’t design; rather, the instructors are simply plugged into a course, given a syllabus and told which text to use.  They may not even create the exams.  This trend, which I find disturbing,  is part of the process of commodifying teaching.  The role of the instructor seems to be limited to presenting content, and often but not always, doing the grading.  While I can see some advantages to this trend (for example, cost efficiencies, tighter rein on what is taught which is important for prerequisite courses) I wonder what it does to learning.  My fear is that it reinforces the transactional notion of education as a collection of facts to be transferred to the student.  This transfer is then stamped with the certification of a grade, credits earned, a degree, etc.  But is it education?

When I was a new teacher, I saw my role as the provider of course content (through lectures), and the examiner to see how much content was assimilated by each student.  I was supposed to be the content expert, even when I taught courses for the first time, or on subjects I had never studied!  In graduate school when I was asked to teach a course whose subject I had no experience with, I was told:  “You’ve studied microeconomics.  This is just applied micro.”  And I did it.  In my first year as a full time faculty member, I was assigned to teach two courses on subjects I had never studied.  That’s just part of what happens at a small liberal arts college, but it’s not ideal.

I now have a broader conception of (good) teaching.   244384064_503e0a152a_mThe most important job of the teacher  is designing the learning environment for a course.  Learning occurs when students interact with and make meaning of the course materials, as they interact with the instructor and other students.  How, and to what extent, that interaction will occur is an integral part of the learning environment design process.  When a student interacts with the course materials, he or she is confronted with new ideas and new ways of thinking.   The instructor provides expert guidance about how to understand the new ideas and ways of thinking.  Other students can similarly catalyze the learning process for their peers by asking questions the student didn’t think of (or verbalizing questions they were reluctant to ask), and by presenting new insights or by providing a more accessible explanation than the instructor did.   A course lacking quality instructor interaction is merely self-study; there’s nothing wrong with that, but it’s not a university course and the learning process will be hampered.

Suppose you were plugged in to teach a course that you didn’t design.  Suppose you believe that students learn best by writing about the subject, that writing requires students to think more deeply about the subject than simply reading the text does.  Suppose that since writing is an integral part of your teaching, your practice is to give essay exams.  Now suppose that the course you are assigned to teach emphasizes content coverage over content mastery.  The administration wouldn’t articulate it that way, but that would be the reality for the average student.  They simply say that you as instructor need to cover the content.  And the content would be assessed by giving a common, departmentally designed multiple-choice exam.  Would the teaching be different from if you had designed the course?  Would the learning be different?  I suspect that students who are brighter, or who learn more quickly would earn better grades in the course you didn’t design.  But would the course be teaching students or merely sorting them?  I’m trying (really!) to keep my biases out of this parable, but I do believe that ownership of the course design matters, that it has an impact on learning. 

When an instructor designs a course, he or she is making decisions explicitly or implicitly about a number of dimensions which define the learning environment. Design palate These dimensions include:

  • To what extent will the instructor tell, coach or model for students?
  • How much of course ‘content’ will be pure content (i.e. facts or findings in the discipline) vs. knowledge creation or research?
  • Discovery: Are students expected to discover insights or will insights be provided to them by the instructor or text?
  • Meaning Making: Are students expected to make meaning or will they be told meaning?
  • Authenticity:  How much of the course work is mere school work vs. real?  (The difference is the extent to which people outside the course care about the results.)
  • Degree of Interaction: What degree of interaction is expected between students and faculty, between students and other students?
  • What is the source of the course materials: Professor vs. Text (secondary sources) vs. Readings/Literature (primary sources) vs. Student Creation? Also, who selects the course materials?  (e.g. Professor chooses the text (or Dept chooses the text) or students find their own information.)
  • How will class time be used?  Lecture from the text or instructor’s knowledge vs. Discussion vs. Demonstration/Laboratory/Learning by Doing vs. Student presentation of content (not for the sake of learning presentation skills but to instruct others.)
  • To what extent will out of class work be structured or unstructured?

Were any of these dimensions things you haven’t explicitly considered before?  What other dimensions can you suggest?

The answers to all of these questions about course design should be based on the course objectives (nominal and actual).  Course objectives might include:

  • Exposure to a subject/ability to be successful through the end of the course (i.e. to learn/remember enough to achieve a certain grade on a final exam, but without any real expectation of retaining or transferring to other contexts).   Few teachers would articulate their goals this way, but many teach and assess this way, especially in general education courses.
  • Introduction to particular skills such as writing, speaking, computing, music performance, artistic performance.
  • Practice in and ability to demonstrate disciplinary analysis (e.g. thinking like an economist, analyzing a problem like an economist would) 

After designing the learning environment, the next 67280931_1568a693ab_m most important job of the teacher is to be a learning guide, that is, to facilitate each student’s learning process as an individual journey.  This requires building (or at least being open to) a professional relationship with each student.  The final responsibility for this journey is with the student, but he or she needs to believe that that teacher is willing, even eager to guide them.  The teacher is, after all, on their own intellectual journey albeit further along than one’s students.  Am I suggesting that the teacher meet regularly, face-to-face with each student?  No.  Rather, the teacher needs to make students believe that he or she welcomes and values the opportunity to discuss course material with them.  Sometimes this interaction occurs face-to-face before or after class or during office hours.  Sometimes it may be a phone conversation, or an email exchange or an IM chat.  Most often, though, it occurs in class sessions when the teacher lectures or leads discussion, and when he or she responds to student questions.  The critical factors are showing one’s own interest in and enthusiasm for the subject, and showing respect for what students have to say, even when it may reveal misunderstanding.  Fundamentally, a good teacher is one who makes students believe that his or her goal is to help them get from where they are intellectually to as far as they can go over the course of the term.

How many students see university education this way?  I suspect that students see ‘school’ as an aggregate thing.  A degree is a series of courses.  A course is a series of lectures (say), which a student is supposed to tune into, and learn at least some of it.  The student sees themselves as part of a whole (the class), or buying into a whole (the ‘standard curriculum for the course’ or perhaps the teacher’s knowledge of the subject), rather than engaging in their own unique journey in which they create their own understanding of he material.  School is like a ski lift.  Hop on, ride a while, hop off.  Students connect for as much as they can or want to, and they learn accordingly.  Their success is defined by their relationship to the whole (did they ride  the ski far enough up the mountain?), not their progress in their own learning (making their own way by hiking up the mountain).

Do faculty perceive their responsibility as teaching students or teaching courses?  Are we taking steps to move closer to or further away from the idea of a personal educational journey when we commodify university courses?  As long as a course lasts a full term, I think it’s the latter.  If we could decompose courses as they currently exist into smaller modular pieces that could be combined in different ways, maybe then commodification of the modules would make sense.

Anyone planning to attend this year’s Education Learning Initiative annual meetings can hear more about these ideas at the session I’ll be presenting.  I hope to see you there!

Teaching as Modeling Disciplinary Practice

Published on August 3, 2009 in Assessment & Grading, Economics, Teaching and Learning and Wikis forTeaching/Learning. 4 Comments

Over the Christmas holidays last winter, as I began to plan for my Spring courses, it struck me that it was a truly fascinating time to study macroeconomics and finance.  Yes, I realize this is much like the neurosurgeon remarking on how interesting the tumor is in the patient he is treating.   Each time I teach a course, I look for a hook that will capture student’s interest.  So I wondered how I might exploit the current economic situation to that end.

One result was the international finance course I teach every two or three years, staff permitting.  This year I organized the class as a research team to explore the global implications of the 2008 financial crisis and economic recession, an event which was unfolding in real time as we studied it.  The dynamics of nearly all aspects of the course, from selecting course materials, to what occurred in class sessions, to grading were different from traditional courses.

Early on we decided that our goal would be to produce an analysis of the “Great Recession,” something that would be useful to people outside the class.  At the time we started, there was no such analysis that we were aware of.  The result is available at http://2008financialcrisis.umwblogs.org .  This is not merely a student project, something only having meaning within the context of the course.  Rather, we set out to create a meaningful piece of economic analysis, every part of which I have vetted and believe in.

About the time we had drafted the analysis, we ran into an arbitrary deadline–the end of the semester.   I asked students what they wanted to do and a substantial number said they were willing to keep working to finish the project.   Roughly half the class continued to work after the semester was over, and roughly half of those were seniors who had graduated.   I believe this is evidence of genuine engagement.

After the semester, we revised the analysis and published it on the website above.  You should note that the global part of the analysis has yet to be posted.  Several of us continue to work on that and hope to have it up in a couple of weeks.

In June I realized that the project could not be completed this summer because the economic crisis has not run its course.   I plan to continue the work next year.  I have two students from the seminar who have agreed to be team leaders, and I am recruiting a handful of promising sophomore economics majors do to the data collection.  We’ll see how far this goes.

Teaching as Coaching?

Published on July 6, 2009 in Assessment & Grading and Teaching and Learning. 7 Comments

erics-double-v-fc

This is another in my irregular musings about teaching, learning and how grading should help rather than hinder the latter.

Does the textbook have a responsibility for student learning?  No, it’s just a source of course content.  Does the teacher have a responsibility for student learning, or is the teacher as expert just another source of course content?  Most people would say the teacher does have a responsibility for student learning, but are most course environments set up that way?  I would say no, not really.

For some time, I’ve been thinking about the parallels between sports and school.  Participants generally think sports are fun, otherwise they wouldn’t participate.  School?  Not so much.

Once the sports team has been selected, the purpose of coaching and the objective of players is to improve performance.  There is regular assessment with clear consequences: players that perform the best get more playing time.  The purpose of coaching is to improve performance.  If the team’s performance doesn’t improve, the coaching hasn’t been very successful.  (Admittedly, this is a collective goal, like a group project.)  But by and large it is successful.  Virtually every team I’ve witnessed improves over the course of the season.

Can we say the same thing about teaching and learning in school?  I’m not sure.  There’s certainly evidence to the contrary.  In a recent article in Peer Review, Bain and Zimmerman retell the story of the experiment conducted by physics professors to see how much learning actually took place in their intro classes.  They conducted pre and post-tests of the some of the fundamental content in their course.  When the professors gave the post-test, several months after the course, they discovered essentially no change in the survey scores, indicating no learning of the fundamental content.  More alarming was their finding that the change in survey scores was unrelated to the grades students earned in the course.  ’A’ students scored better on course exams and other graded assignments, but they didn’t learn the fundamentals of physics any better.  Bain & Zimmerman argue that this result is not limited to physics but applies to all disciplines.

In school, when a student receives a poor grade, they usually feel bad, even guilty, and they often blame the teacher.  As teachers, we tend to discount the student, thinking ‘they failed’  Our emphasis is on delivery of the content, more than on coaching over and over to help the student get it right.  There are some exceptions to this: composition courses where students write and get feedback repeatedly, and teachers who go out of their way to provide recurrent feedback to students who can’t seem to get it.  But this is not the norm.  Usually, the content drives the schedule, and students are supposed to get the content and move on to the next topic.

I think that teachers teach with the expectation that successful students will earn an A.  Anyone less than that, isn’t really successful.  We teach the content; we assess learning of the content.  At that point it’s sink or swim.  The expectation is that the good students will get it, and the rest, well, they’re not good students so the fault lies with them.  We claim to offer help to students who don’t make the grade.  But we don’t really expect that everyone or even very many students will come to our office hours.  By and large, we expect that students will do the lion’s share of their learning, their remediation on their own.  We test them, we give them the correct answers and we expect them to figure out where they went wrong on their own.  There’s a certain contradiction in thinking here.  At one level, we act as if everyone should get an A.  (We often teach to the level of A work.)  But we don’t really believe that, and in an important sense, for those that don’t, we wash our hands of them.

Compare this with sports and coaching.  In sports, there is little expectation that players will perform perfectly on the first attempt.  Once players have made the team, the purpose of practice is not to delivery content (how to field a grounder), though there’s a great deal of that done.  The purpose of practice is to provide repeated opportunities to improve performance.  The objective of coaching is to help players improve from where they are.  After all, if the goal is to improve the team’s performance, helping each player to improve will contribute to that goal, even if no players become all-stars.

In school, grading tends to be a sorting mechanism of whittling students down. The grade is perceived as a judgment about one’s personhood, rather than as helpful feedback.  Not surprisingly, students push back against grades.

In sports, assessment is more directly connected to performance.  The goal of learners and teachers is to build one up, not document how one falls short of perfection.

I wonder if teachers could learn from the sports approach.  That is, if we are more interested in learning than sorting.

Not part of your permanent record!

Published on September 27, 2008 in Assessment & Grading. 4 Comments

This is another post where I explore and struggle with the notion of grading.

This week I reviewed the first substantive assignment I asked my intro students to write this semester. It was actually assignment number three, where the first two took effort but not much thought. This one asked students to complete a sophisticated task: the apply an economic theory to a real world situation and, after identifying the facts revealed by the theory, to draw some normative conclusions about the situation. I have used this assignment for years and know what to look for in their answers. As I read the essays, I wrote substantive comments for how the essays could be improved, raising questions that would help students take their thinking to a higher level. Only after my reading and writing of comments on all the papers did I start to think about assigning grades.

A not-insignificant number of students failed this assignment. That is, they approached this as a non-economist would, going with their intuition rather than letting the theory reveal the answer to them. Should I give these students an ‘F’ on their essays?

I think it depends on the purpose of the grade. Is the purpose of the grade to make a summative judgment about what the student has achieved on this assignment? Does an F imply earning less than 60% of the total credit on the assignment? I don’t think it should. For one thing, this assignment isn’t about learning content, where learning 59% of the facts isn’t a passing grade, but learning 61% is. Rather, the assignment, applying a theory to derive insights about an issue or problem, is the fundamental goal of undergraduate economics education. It is what economists do. It is far from trivial, and students in the intermediate theory courses regularly show they haven’t grasped it yet. So the objective is not to score the degree to which students fully demonstrate this task, but to help them learn how to competently go about the task.

I have another concern. There’s something about a formal grade and the way that students usually interpret grades that gets in the way of learning. I think this is especially the case for first year students who often have always excelled grade-wise. Getting even a C on an assignment can be a rude awakening to college. I think that the feedback teachers provide on assignments, particularly early in the term, should provide operational information about how students can improve. Traditional grades don’t necessarily do that.

We need to provide opportunities for students to take intellectual risks, but where the consequences of failure are minimal. We need to encourage students who have taken the right approach, a very sophisticated approach, but one which they haven’t executed perfectly, to continue to work in that direction.

What I propose for homework assignments is focusing on the approach students take and (largely) ignoring the details of their reasoning. What this implies is affirming the student who’s taken the right approach but whose product might not be refined enough to earn an A. It also implies providing a signal to the student who is fundamentally taking the wrong tack, but who may have provided enough ‘content’ to avoid an F. In other words, it’s not about assessment of where they are, but the direction in which they are heading.

In practice, I give students credit for a genuine effort at completing the assignment. (On rare occasions I give no credit if it’s clear that the student didn’t put in real effort.) I evaluate their effort using a three level scale which is easily assessed: On the right track, Not on the right track, or Sufficiently well done to be nearly perfect. When computing final grades for homework, I come up with a holistic judgment based on the pattern I see in their assignments graded this way, from which I determine a traditional letter grade.

Spam prevention powered by Akismet