Monthly Archive for January, 2006

Knowledge Management?

The Wall Street Journal had an interesting article today entitled, “Companies Struggle To Pass On Knowledge That Workers Acquire.” The website requires a subscription but you might be able to get the article here. The article describes a common problem in business today, namely how firms lose the knowledge learned by workers on the job when those workers leave their position.

We’re all “knowledge workers” now. But few organizations have figured out how to share knowledge among employees, or to pass it on when employees leave or change assignments. … Business gurus have offered dozens of potential solutions. Most involve technology, like asking employees to submit tidbits of expertise to a database that other employees can tap. But few of these efforts have produced big payoffs.

Describing one initiative to capture that knowledge, the author notes, “[T]he technicians were reluctant to submit tips. They didn’t “find it natural to write down what they knew.”

As I read this article I couldn’t help thinking of the parallels with academia. It seems to me that academics are even more reluctant to talk about their craft than most professionals. And yet teaching really is a craft or an art. It’s not something you can learn from a book. The sad thing is that most of us don’t really know how our colleagues do what they do in the classroom. It seems ironic given that we’re in the business of public speaking, just as long as its not too public, I guess.

There ought to be a serious way we could capture that knowledge and share it with others. (Sounds like a job for our Teaching Innovation Program.) But every idea I come up with tends to be trivial–like asking faculty to write down their teaching tips and putting them on a webpage. The best teaching insights I’ve found to date I’ve gleaned from reading teaching blogs, but few of my colleagues at Mary Washington blog.

I’m not too optimistic about finding a solution to this. As the article notes,

We don’t necessarily understand enough yet about optimizing the conditions for knowledge work, even though we’ve been doing it for 25 years,” says Hadley Reynolds, research director for the Delphi Group, a consulting firm. “Most organizations are still managing as if we were in the industrial era.”

That goes for academia, too.

Does Research Inform Teaching (or vice versa)?

Last Spring I used a wiki to teach a senior seminar in macroeconomics. The wiki provided a parallel out-of-class universe for the course. At the time, I knew only the basics of what wikis were about, so we were very much “learning to fly” which was the metaphor for the class experience. I gave a presentation about this experience at the 2005 UMW Faculty Academy.

In a little over a week, a couple of students and I will be giving a revised version of that presentation at the 2006 ELI conference in San Diego. Over the last several months we have been researching wiki theory and usage in preparation. Not surprisingly, I think I’m finally beginning to get wikis. In reviewing what worked and didn’t last year, I’ve come up some of ideas about how to make better use of the wiki in my senior seminar this semester compared to last year. Here are a couple of them.

*Collaborative List Making—When I first heard about using wikis to create lists, I admit my first reaction was underwhelming (sorry Brian!). I no longer feel that way. My seminar this semester is titled, “Contemporary Economic Issues.” To encourage the students to buy in to the course, I decided to let the students choose the issues we would study. To that end, I asked each student to bring three choices to the first class. I sent them the list of past topics from the course, but told them they needn’t limit themselves to those topics. The first use of the wiki was to post their choices. I then sorted and collated the topics to determine the top choices. The most popular topic and the first one we will investigate is globalization. The next task will be to develop a list of references for the topic (another list). This list will turn into an annotated bibliography when students post their notes from the readings.

*Wiki as process vs product—One way that wikis are superior to traditional webpages is that they have a certain organic nature. What I mean is the ability to use and reuse wiki postings for different purposes. I’m probably way out of my element here, but if you think of wiki postings as data records, then the wiki can be seen as a database, where we can sort and organize the records in different ways merely by creating new links. In creating the ELI presentation I found myself using pages of notes developed during the research process and turning them into products—pieces of the presentation, just by creating new links.

I plan on learning a lot more about “power uses” of wikis this semester. Feel free to follow along with my seminar.

Good Class Today

Today, I had a very successful class in both sections of my intro course. The material I presented was my take on the “economic way of thinking” (the methodological approach used by economists), which I boiled down to six principles of microeconomics. I put the six principles on powerpoint slides, which were linked to the course syllabus. I encouraged students to listen rather than write down the principles since they were available on-line. When I use powerpoint in class, I tend to be a minimalist, using the slides only as an outline of my remarks. I think that provides structure for the students, while at the same time they still have to pay attention to what I have to say.

I fleshed out the principles in detail, going well beyond what was on the slides, providing examples that I worked thru on the board. I made two digressions from the powerpoint to webpages off-campus to collect some real world data to illustrate points I was making. Then at the end, I played a podcast from Slate.com which illustrated rational decision making. For homework, I asked students to review the podcast, which was also linked to the syllabus and to try to explain the story in the context of economic rationality.

My point in blogging about this is not that bells & whistles make a better class experience. Rather, the guiding principle was something that I learned at a teaching workshop in 1994–that students learn more effectively from mini-lectures interspersed with other activities. Additionally, since students have different learning styles, I hoped to reach more students than I would have with a traditional 50 minute lecture.

Preliminary Evaluation of the Experiment — Part 2

In the previous posting I reflected on the changes in course content I made last semester. Here I want to comment on the meta activities.

One of the things I did over Winter Break was to analyze the frequency of student participation in the two types of meta activities: the abstract metas and the on-line quizzes. I describe the goals of the abstract metas here.

The results were a bit depressing.

Abstract Metas:
* Nearly 60% failed to participate (59% submitted 0-2 of 12 assignments).
* About a third participated moderately (34% submitted 3-6/12 times).
* Only 10% participated regularly (7-10/12 times).

On-line Quizzes:
*More than a third failed to participate (35% submitted 0-2 of 11 quizzes).
* About 40% participated moderately (submitting 3-6/11 quizzes).
* One quarter participated regularly (7-11/11 quizzes).

Still this should provide enough variation to obtain meaningful statistical results, when I finally get around to estimating them…

Preliminary Evaluation of the Experiment — Part 1

I finished the first semester of my experimental intro course. Here are some preliminary thoughts about how it worked out.

The original intent was to change the course in two ways, first by modifying the content of the course, or more precisely the content of the class sessions, and second by trying to teach students metacognition to enhance their learning.

With respect to the content aims, I was fairly successful at not lecturing on basic content from the text, but rather using the class sessions as laboratories for non-lecture activities. Looking back at my notes I spent only 7/42 class sessions lecturing, including several mini-lectures (comprising only part of the class period). I also did well at keeping on a clearly defined schedule. The result was that I “covered” more material than in the past few years, though at several points I worried about possibly sacrificing depth for breadth of coverage. That wasn’t the plan. The plan was to add depth of meaning thru metacognition, though not depth of coverage. For some topics, I clearly didn’t take the class into the same depth as I had in past semesters, leaving it for them to do on their own.

What did I accomplish in lieu of lectures? I added a few more class discussions as well as a few more group activities. What about new media? Here I wasn’t as successful.

Audio – I identified a few items I could have used, but mostly after the fact. I only got one podcast into the classroom.
Images – I used images a few times where appropriate. More precisely, for a couple of topics I created slide shows of images to illustrate the material. I missed one opportunity (The Great Depression) due to lack of prep time, that I have since rectified for next time. This is at least an improvement.
Simulation — I had tentatively identified two simulation exercises that I thought would work well. When I got to them, one turned out to not be appropriate. The other required a fee for usage, which I didn’t feel I could impose on the students at the last minute. (The fee would cover access to the entire site with more than a dozen simulations—I only wanted to use one.)

In retrospect, I identified one major problem with the course: The text we used didn’t match my course content enough. Especially during the middle of the course there were large differences between how the text covered the material and I how did. The largest discrepancy was that the primary macro model I taught was relegated to an appendix in the text. The one the text focuses on I didn’t teach. There were sound reasons for this–that model is unnecessarily complex and misleading to the students. I choose the text because it was one of the few that cover the simpler model. Most don’t cover it at all. I was able to teach everything I needed to teach more simply with the model from the appendix. Several times, I told students that we would cover the material presented in the chapter but using the model in the appendix. Nonetheless, many students were still confused. (See earlier posting on students studying the wrong stuff.)

I wonder if I tried to accomplish too much this semester? Did I try too many innovations: teaching metacognition, changing the content of the class sessions, and completing an earlier experiment on critical thinking? This semester I have more modest goals. I have completed the critical thinking work. I will add new media where I can, but I plan to focus on teaching metacognition.

New Insight on Blogging as a Tool for Learning

Martha just posted a very thoughtful reflection about how blogging can enhance student learning. The crux of her argument was this:

But more interesting to me … is the way in which blogging might have encouraged me to more thoroughly “fix” the knowledge I was generating. If these entries had been a part of a larger blog that recorded the intellectual development that I was experiencing in graduate school, I might be able now to recall how I had gotten to the words contained within them. By recording the intellectual process that I was going through, I might have made it more real for myself and, thus, more memorable. As it is, I can’t recognize any patterns of application of these ideas in my life today–perhaps because, I “lost” them in some sense.

How many times have you figured out something, that you meant to follow up on, but never got around to? But Martha doesn’t stop here. Rather, she seems to be illustrating a way of connecting classroom learning with the broader notions of meaning making and lifelong learning. Now there’s an idea.

A Modest Proposal

In earlier postings, I noted that I’ve been reading a very fine book: Educating the Net Generation, published by the Educause Learning Initiative early last year. I recently finished Chapter 8: ”The Real Versus the Possible: Closing the Gaps in Engagement and Learning” by Judith Ramaley & Lee Zia.

I found this to be a very powerful essay. I don’t, however, envision this as the authors do, as necessarily focusing on cyber-education. To me it is just education (real school?). Every generation of educators teaches with a technology, and every generation thinks about transforming education to make it more real. This is just the latest installment, albeit a thought-provoking one.

Ramaley & Zia begin by posing the problem:

Although we know a lot about learning and continue to learn more, there is a gap between what the education research community and the learning sciences have discovered about learning and what most of our faculty know or practice.

How much of your graduate education focused on pedagogy? How many of your colleagues have read any of the literature over the past decade on teaching and learning? Of those, how many have made any substantive changes to their teaching approach?

Next the authors dispose of a straw man, clarifying what is the same and what is different with the latest technology.

[N]ewer forms of interactive technologies… are not meant to replace traditional forms of learning. Rather, they enrich traditional forms of learning and serve as links between active and passive, individual and group, and transmission and generation of knowledge. The criteria we apply when assessing the quality of the material we offer will, at one level, resemble the standards that the academy has set for any intellectual work: originality and importance, thorough grounding in the field, clarity of goals and expression, effective use of materials, and ethical handling of material and ethical approach to the user [emphasis added]. However, the standards for presentation in these new media and formats will be different.

The essay then moves on to the main event: What it will take to succeed in this transformation of education? The answer: Educators must change their practice from teaching students to assimilate knowledge (e.g. what are the principles of economics?) to teaching students to perform as disciplinary practitioners do (e.g. doing economics).

Now it’s probably true that at present our very best students achieve this, if only by accident since this isn’t generally what is taught. But Ramaley and Zia are arguing that this should be the goal for all students (and for all faculty).

How do we get there?

The very thought process that leads to discovery and understanding in a particular field can be exposed and modeled for students, who can then have an authentic experience within the discipline.

In electronic exchanges, faculty members are free to be experts (for example, a physicist, a biologist, or an historian) and to draw their students into the ways of thinking, examinations of ideas, and forms of proof that are the intellectual basis of a field. In addition, original documents and fresh research data are readily accessible on the Web.

Students can find material that challenges the faculty member’s worldview and expertise; they can uncover stories and research results that the faculty member has never heard about. It can be uncomfortable when the instructor no longer controls the subject matter the students will use.

The instructor can model intellectual work, exposing through electronic means thought processes and realities—the blind alleys and sudden bursts of clarity—that we all experience in our search for understanding. For many, this is unnerving; control is lost over both the interaction and the material. For others, it is a true liberation. For everyone, however, it can provide a much more immediate and authentic experience of inquiry than most classroom interactions can offer.

The important insight that will guide our exploration of the value of interactive technologies is that a user of digital information is certainly being asked to be active, but is probably not being asked to be reflective.

And to truly educate we need to change this.

The most powerful effect of cyberexperience may not manifest in the things people do on the Web or with broadband communication, but rather in how they think and in what they expect from education. People who innovate and create in cyberspace likely will not sit still for a lecture.

Is anyone out there listening, though? I worry about the lack of substantive conversation on my campus by the majority of faculty about what we are about, and whether it’s what we should be about. From my perspective, most of those that even consider this question assume that their teaching is at least adequate to the task of educating this generation of learners. For the rest, it’s not even an issue. To paraphrase one professor, “we mustn’t forget that the paramount goal is to ‘cover the material.’” Perhaps the reality is that we’re going to have to wait for the next generation of teachers—those that were net generation learners.

Reflections on the Interactive Quizzes

During the last week of classes, I noticed that a number of students were doing the interactive quizzes to study for the final. How do I know this? Well, because several of them were taking quizzes they’d already received credit for attempting early in the semester, so there was no “credit” at stake here. One student emailed me the following:

Professor Greenlaw, Wow, I did horrible on that test, I guess I’ll have to study that section some more. Most of the time I was torn between two and picked the wrong answer, but I’ll study [Chapter] 16 some more.

Clearly some students found the interactive quizzes useful. Indeed, on average students submitted more interactive quizzes than metacognitive activities. I certainly found them useful for determining how many students were doing the reading or not. I am assuming that if they did well on the quiz they probably read the chapter, though conceivably some may have taken the quiz to see if they needed to read the chapter. That could explain the handful of the really low scores.

Notes on the Remaining Meta Activities

Here are my rough notes on the last nine meta exercises done by my students. It’s probably accurate to characterize this as raw data, so this posting is mostly for me.

Meta 4 — N = 18/52 submissions
This is the meta we roughed out in class on the day before Fall Break. It looks as though I received fewer submissions than the number of students who were in class that day, though a few submitters weren’t in class. Also, the submissions don’t appear to have gone substantially further than we went in class. This is disappointing. Could the students have decided that our class outline was sufficient to study from?

Meta 5 – N= 8
Few submissions, but a third very well done. Also, 5/8 from 10am; only the two foreign students submitted from 11am. This was the topic I did the day before Fall break. I mostly referred the class to my lecture notes on Blackboard. The absent students may have missed this reference & topic. But what about the others? Was the fact that we didn’t spend class time on this short topic meaningful? Students apparently didn’t make the effort to do this work on their own. 2/3 of these submissions were so poor that I couldn’t differentiate them from meta 6.

Meta 6 – n = 4 + 5 = 9
Submissions were only adequate on this assignment. Lots of concepts identified as institutional facts/findings. Lots of concepts & IFFs identified as theories. What is the hypothesis for these theories? Not enough justification of the selections students made.

Meta 7 – n = 3 + 4 = 7
Generally good submissions. Submitted by people who have submitted most metas previously. Some students continue to have difficulty defining metas by chapter instead of by topic. I’m beginning to wonder if this is more than a trivial difficulty since it suggests perhaps that they can’t think outside the chapter. The two students who do this the most are foreign students—could their English listening skills to what we do in class be the culprit? But for Meta 5, I didn’t spend any substantial class time, merely referring students to my lecture notes which I posted on Blackboard, so this hypothesis can’t explain Meta 5.

Meta 8: n = 4
Pretty good submissions for the handful of students who did them. This was due right before the second exam, and I blogged on this earlier.

Why the declining numbers of submissions?

Meta 9: n = 9 + 4 =14
This is the first meta after the second exam. This is better than the last one, but why still so few. Are they too busy with other work as the semester winds down? I see this topic (Economic Growth & Business Cycles) as primarily focused on theories: The income-expenditure model and the economic growth model. Students pegged it as conceptual or focused on findings. The major shortcoming of these submissions was the lack of emphasis on theories. Many students included no theories in their submissions. The rest choose one, but not both. Usually they chose the growth model. This makes a certain amount of sense given that the text chapter focused exclusively on economic growth—there was no chapter on business cycles per se, even though in class we emphasized cycles (in the form of the income-expenditure model) slightly more than growth. In general, these submissions weren’t very strong, though my sense was the students were trying. Five submissions were from students who submit regularly. Two submissions were from students who haven’t. Perhaps these guys were motivated by my speech. The rest were in between.

Meta 10: n = 8 + 3 = 11
Most submissions were quite good, clearly genuine attempts to do a good job. One was trivial. Most submissions were from the early class, where I believe the complaining students came from. This is the last week of class, so people are running out of time. This meta was due Wednesday. The previous was due last Friday. The last will be due this Friday. This compacting of deadlines probably hurts too. Notably, five submissions were restarts: students who haven’t turned anything in a while (except chapter 9 for some).

Meta 11: n = 2 + 1 = 3
These were pretty good.

Meta 12:
One person submitted this despite the fact that we never got to this material.

Data for the Previous Posting

Email to Students After the Second Exam:

I am interested in your reactions to the second exam which you took yesterday. What did you think of the exam? For example, several students mentioned to me that “they studied the wrong material.”

If you had any reactions, like that, or others, I’d really appreciate your letting me know what they were.

Agreement with the Sentiment: 10 responses

“I concur with that students statement. It felt like all the stuff I studied for was not on the exam either. There were no specific questions about the graphs or the equations we learned about. Hopefully I did okay.”

“In terms of the exam, i thought it was hard but if it was easy that would be no fun right? The second exam was different in style than the first exam but at the same time both exams had two totally different types of material- in terms of linear and abstract. In addition, the material on the second exam was more lecture based material so naturally the second exam would be focused on questsion based on lecture. However, studying purely by lecture notes in this class is kinda of hard, because it is hard to look at the book for further material. The exam was hard but i think it was fair. All the questions asked i am sure was material that was went over in class. I hope this doesn’t mess up any curve you had planned, if you had planned one, because i am sure i am going to need it!â€?

“I do feel that I studied the wrong material, or at least was unaware of the types of material I should have studied. For example, I was definitely caught off guard by the more current economics questions, and while I felt like I studied and knew the material we had covered fairly well, I was still unsure of the answers to a lot of questions.�

“Well I didn’t feel as though I studied the wrong material as much as that I studied the wrong aspect of the material. I was familiar with 90% of the concepts asked about, just not with the part you where asking for. For example I felt I had a strong grasp on everything to do with the classification of money, but with the $500 million dollar question I was at a lost.â€?

“I think most of the problems I had were that I studied more of the definitions and classifications of things than I did practical applications. I felt like I knew what the question was about, and what factors affected it, but couldn’t extract the ‘big idea,’ if that makes any sense. I don’t know if that means I studied the wrong material, or just studied wrong. Overall, I didn’t think the test was too terrible, but then I haven’t seen my grade yet! I hope this helps.â€?

“To prepare for the first and second exam I primarily studied the online quizes. This seemed to really prepare me for the first exam, but it seemed on the second quiz more of the questions were from notes than online quizes.�

“I felt like I studied mostly the right material. I thought that the last test had very specific questions so I tried to remember a lot of specific details. The questions on this test, though, were broader I guess so I just had to think of all the little things and put them together. I did over-study the last topic, what determines economic activity. I thought most of the questions would be on the big equation but barely any were. The questions that have answers I and II always mess me up and i thought they were the hardest questions on the test.�

“I did think that the second exam was a lot harder than the first, but I just figured that that had to do with it being all new material for me. I did think, however, that esp. with the income expenditure model that I knew the material from class notes and what not, but that the questions on the exam were different than what we went over in class. Maybe it was just me, I dont know.�

“I thought the exam wasn’t what I have expected to be. I expected be more technical and in-depth. For example, question 2 asks 2005 Nobel Prize in economics, which is irrelevant to what we have studied in class. I knew that Milton Friedman received a Nobel Prize in economics but this was the only hint I could get. Other four choices were guessing work. I am really disappointed in questions that tells you to determine I and II category if it is false or true. It is ambiguous. I am not harshly criticizing the exam but I believe exam should be more text based and techinical.”

“I definitely didn’t understand the material the way I thought I had. I had difficulty applying things.â€?

Material that Wasn’t Covered in Lectures or Texts: 5 responses (includes 1 above)

“I thought the test was fair on the whole, as there were concepts from the book and some from class. However, I found that several questions were either from material that was not discussed in class or not from the textbook. The question about the Nobel Prize, for example. As well as the question about Laissez-faire and Policy Activism. I have been to every class this semester and read every page assigned to me and I do not remember talking about either of those things in class or seeing them in the text. While the test did have many concepts that were covered, on a test with only 30 MC questions, I think it would be ideal if all the questions came from class or from the text. Like I said, for the most part, I felt as though the test questions were fair, however on a test where every question counts, every question should be “fair.” I don’t necessarily think I studied the wrong material, as I reviewed and re-read the book and notes from class, as well as doing the exercises from the workbook and the online quizzes for the chapters that were going to be on the test according to the syllabus. I simply think there were a few questions that were unfair in the sense that they did not encompass material covered either in class, in the book, in the workbook or on the online quizzes. While the test was relatively fair, there were several questions that were exceptions, and on a test where if 6 questions are missed your grade is a B-, every question counts and thus every question should relate to concepts, facts or theories that, according to the syllabus, we were supposed to know about for the test. Thanks for asking for my opinion, and I really appreciate you reading this long-winded e-mail!”

“The exam was, like you told us, fairly difficult. However, I thought I did study the right material. There were some questions, though, that showed up on the exam which we did not really discuss. For example, the questions concerning who won the Nobel Peace Prize, and whether Bush or Kerry believed in laissez-faire or policy actvism. These questions threw me off greatly. These were the only problems I encountered. Other than that, the content of the exam was what I expected. Thank you for allowing me to share my input with you.�

“I thought some of the questions took me by suprise like the ones about john kerry and george bushs views on economy, the nobel prize winner question, the fiat dollars, and the amount of time we spent on the aggregate supply/demand and the expenditure formula and so forth wasnt really reflected on the test.�

“I really did not keep up with George Bush and Kerry’s ideas on the economy, so that question really through me for a loop. Besides that the exam wasn’t as bad as I thought it would be.â€?

General Complaints: 2 Responses

“Thank you for asking for our feedback and taking an interest in our opinions. I felt VERY unprepared for this test. It seemed that the heaviest information presented in this introductory class was crammed into an inadequate amount of time for successful application and digestion by those of us who are not economists or economic thinkers. In preparation for this test we were given no formal notes, information or a class review. We were left to abstract information from 5 topics, 70 pages in the book, 30 pages of handwritten class notes and a few handouts. That is alot of brand new material, theories and concepts to disect and decifer to what will be applicable on a 30 question test. I felt we were not given enough time or support to absorb, understand or apply all of this new information. I walked in on Wednesday feeling set up to fail.�

“I thought some of the material on the test was very tricky. I personally felt that there were a few trick questions on the exam. For instance, the true and false question about employment. I can’t exactly remember it, but it was something like ‘If unemployment increases, then the amount of jobs decreases.’ Something to that effect, but it didn’t specify whether you were talking about jobs AVAILABLE or just jobs in general. I’ll have to look at it again…so I can remember exactly what I was thinking haha. I studied a very long time for this exam (not just the night before) and there were definitely some questions that I know I got wrong. I don’t have a great feeling about it :( “

Generally Favorable Comments: 7 responses

“I felt that the exam was fair and I didn’t feel like I studied the wrong material. I just read over the chapters from the topics online and studied my notes. Overall, I felt that the exam was fine and I hope I did well on it.â€?

“I think the test was fair…straight from the notes and from the additional material that you gave us on blackboard. You had to think about the questions and how they apply the theories instead of just regurgitating facts. However, the one about the noble prize in economics through me off, but it was just something i did not know.â€?

“XXX and I studied from your Study quide questions after each chapter… Alot of what was on the test came from those sheets. The test was a bit challenging… but that is what they are for. Most of the questions I was able to read the question and immediatly see the answer that I knew it was, others I had to think more about. Then again I have no idea how I did on the test. On the first test I had thought I did very well when actually I didnt do as well as I had thought. The test seemed fair though.â€?

“XXX and I did the study questions at the end of each course topic explanation page. Nothing seemed too out there. I also skimmed the book. To me it didnt seem like the wrong material, but we’ll find out tomorrow when I get my test back.â€?

“The exam was pretty much what I had expected. The questions seemed to come mostly from material we covered in class. I studied my notes from class and also the assigned chapters in the book that supplemented the lectures. Overall, the questions on the test seemed to be from information that I had seen before.�

“I didn’t think that the exam was that bad. I’m guessing that we did poorly or else you wouldn’t be asking… The only thing that I can say is that it seemed a bit more abstract than the last one, but I think that’s the nature of the material that we covered.â€?

“Well, I thought the test was pretty fair. The only thing that I thought tripped me up were the questions that said “Think carefully!” It made me second guess my answer and made me unsure of myself.â€?




Spam prevention powered by Akismet