On the nonlinear way my mind works

I always get a thrill on those occasions where, for a while, many things I read or hear or see seem to be on the same topic. Or maybe it’s just my mind seeing connections. This is a longish post, so bear with me—there is a punch line.

Earlier this semester, around mid-term, I was in New York with several students—each year I take students to present their original research at a professional conference. The undergraduate sessions, which I help organize, are sponsored by Issues in Political Economy, an undergraduate journal of economic research (broadly defined). Anyway, when I saw and read Gardner Campbell’s post Conceptual Frameworks: Some Thoughts, I knew I had to share it with my senior seminar.

{Gardner’s post was a contribution to #OpenLearning17, an open online course that is running this Spring as part of the AAC&U’s Faculty Collaboratives Initiative, which I’ll explain in more detail below.}

This year’s senior seminar follows a model that I’ve used before. I identify an interesting question to which I don’t know the answer.  I organize the class as a research team and we spend the semester exploring the question and figuring out an answer, which we publish online. The process is loosely based on backwards-design: We start from the research question and ask “What do we need to figure out to be able to answer the question?” We divide up responsibility for those things, and the students research them. We report out in class and make note of what we’ve learned. Then we repeat the process in greater granularity. Finally, we draft a report of what we’ve learned (which process we’re doing right now). In lieu of a final exam, I ask students to reflect on two questions: What have you learned, and what have you contributed to the class’ learning? I ask them to provide evidence from their own work (e.g. blog posts, written reports, presentations to the class, contributions to the final report) to support their claims.

This year’s group has great chemistry and I’ve been very pleased with our work so far. After reading Gardner’s post, I emailed the class, told them we would move what we had planned for Monday to Wednesday, and instead I asked them to read Gardner’s post and be prepared to discuss it on Monday.  In class on Monday, I posed three questions to the group:

  1. What is a conceptual framework?
  2. Why should a course have a conceptual framework?
  3. What is the conceptual framework for this seminar?

I have to admit that I didn’t know if the discussion would take, if it would last five minutes or the entire class period, I just felt it had potential. The discussion turned out to be extraordinary. Every student had something to say. Four students had already blogged about Gardner’s post before class without any prompting. I want to highlight the contributions of two students in particular:

B. is the only student in my seminar who is not an economics major. But she keeps taking my courses. She spoke up saying that she was the kind of student who liked to know exactly what was expected so that she could exactly meet the instructor’s standard. She said my classes make her uncomfortable because while they may provide a broad framework, they don’t specify exactly what is expected. But for some reason she has been willing to take these courses that make her uncomfortable, because she has learned that she will get something important out of them.

A. is the only student in my seminar who has not taken a course with me before. In her comments, she quite simply went off on what’s wrong with education today, or as I describe it, the game of school, which she asserted encourages good grades and test scores to the detriment of real learning. She went on to say that the courses where she has learned the most were those that set broad guidelines and then expected students to figure out what to do with them.  We might call those courses examples of “real school.”

Why am I bringing up this seminar? Because by not seeking to convey a specific body of content, and by not knowing the outcomes in advance, the seminar employs a fairly open pedagogy. More on this point later.

A few days after my seminar discussion while meeting with students in my online intro class, many of whom are adult students who need my class for their degree programs, I witnessed a different perspective. These students have jobs and families. They don’t have a lot of free time. Many of these students have a very specific goal in mind, and it’s not education, the way I think of it. They are taking my course to meet specific degree requirements. They are pursuing their degree program because they see it as instrumental in reaching a career goal. In some cases, they merely need a college degree. These students want to have a rubric. They want to be told exactly what they need to know to get the grade: A, or B or C. They don’t want an open learning experience.

They seem to be asking:

Can we just skip the process and go right to the learning that’s going to be on the test?

To be fair, they are not asking for the answers; they just don’t want to spend scarce time on things that aren’t important, which they take to mean things that won’t be on the exam. But what if there are no exams in a course?  Does that mean there’s nothing worth learning?

It’s hard for me to fault these students. They are doing what they’ve been socialized to do. They are trying to win the game of school. They are paying good money to get a job, career advancement, etc., and in many cases they have a clearer idea of what they want out of college than my traditional-age students.  Some of these students are really good, A students. They have great potential to be educated, but that doesn’t appear to be their goal, at least not in the way I mean it.

What about the many students who start college without a clear idea of what they’re here for, what their mission is, what they want to accomplish at school. Some of them find their way and flourish.  Most of them graduate.  Some of them don’t. What responsibility do faculty have here?

Perhaps it’s unfair to compare introductory with advanced courses, but I am left wondering what faculty should do.  University students are adults, after all.  Should we respect their choice and go on with our work?  Or should we assume that students don’t know a different model of school, and then offer them one?

That brings us to general education, and the AAC&U Faculty Collaboratives initiative, of which #OpenLearning17 was a part. Over the last decade, the AAC&U has done terrific work at defining what college graduates need to know and be able to do to become contributing members of society, both in terms of undergraduate major programs and especially general education. This work is largely based on research in the learning sciences, something that few university faculty learned in our graduate programs. The work includes the so-called High Impact Practices, which have been found to correlate with student success.  The AAC&U has also developed guidelines and rubrics for assessment of these goals. So the purpose of the Faculty Collaboratives project is to introduce faculty to these practices, demonstrate how they enhance student learning and success, and encourage faculty to try them out.

In my view, general education courses traditionally haven’t been a good entry point to real school. I’m not suggesting that there aren’t some courses that do the trick. But consider all the general education courses that are primarily designed as introductions to a major. Or the introductory courses which are designed to be gatekeeper courses. These are not courses designed to pique students’ curiosity to think deeply about how disciplines help learners make sense of real world issues and problems. They tend to be about covering content, lots of content. The end result seems to be that students see general education courses mainly as hurdles to be gotten over, impediments to their getting to their real curricula. (It’s not just students with this problem. Think about the language that faculty advisors sometimes use.)

General education courses are not merely courses we are obligated to teach for the institution so that we can teach the upper level courses we want to teach and so we can do our scholarly activity. They are, or should be, important entry points for real school. Students shouldn’t have to wait until senior seminars to get a glimpse of what real school can be.

We can learn how to more effectively teach Gen Eds by incorporating what the Faculty Collaboratives initiatives have publicized and promoted.  Liberal arts and sciences institutions should have a particular advantage here over larger universities with large lecture courses. Compare a research university where only the best undergraduates have the opportunity to conduct research with a faculty member, with a small liberal arts college where all graduates are expected to have done so.

But we can take this a step further if we combine general education with open educational practices. Open education, which has been the subject of #OpenLearning17, is a pedagogical approach that offers extraordinary promise for those willing to try it.  By addressing real world problems that students care about, by giving students agency, by showing students that what they are doing matters, by connecting students to outside audiences, experts and novices who are genuinely interested in what the students have to say, open education and its practitioners can show the power that real education provides to change the world.

That’s my take away from the Virginia Faculty Collaboratives project.

 

Posted in Uncategorized | Tagged | Leave a comment

Using Hypothes.is to Help Students Read Scholarly Papers

One of the things I’ve learned from #OpenLearning17 has been the ability to use Hypothes.is to collaboratively read something online.  Hypothes.is is free software that allows one to annotate any webpage as one is reading.  The annotations are a layer “on top of” the webpage.  They do not modify the page itself.  While the annotations are reason enough to find Hypothes.is interesting, the software allows one to share the annotations with anyone else using Hypothes.is.  But wait–there’s more!  A group of people can read the same webpage and annotate it in parallel, each seeing each others’ comments.  This raises some interesting possibilities.

In my upper level courses, it’s not unusual for me to assign students to read scholarly journal articles.  This is not trivial.  Economics journal articles are a particular type of technical writing.  It takes practice to figure out the format of journal articles, but more importantly, the argument contained in them.  This is especially true when the argument is mathematical, or when the argument includes statistical analysis, as are often the case. I imagine professional writing in other disciplines is similarly challenging.  I assign these readings and hope for the best.  When I discuss the articles in class, I ask if there are any questions, and I ask students what they made of the articles.  A few students ask questions or express opinions.  Most are silent, perhaps because they found the task too difficult.  They can’t even say what they didn’t understand.

What if I told students we were going to read the article together as a group?  What if I left signposts in the form of annotations to guide the reading?  What if I asked students to identify things they didn’t understand and asked others to respond if they could?  I suspect it would be a different experience than my previous practice.  It’s possible that students would get more out of it.  At a minimum, it would harder for students to pretend to do the work when they’ve left few or no comments.  I think this is worth trying.

I wonder what other possibilities there are for using Hypothes.is in one’s teaching?

Posted in Uncategorized | Tagged | 1 Comment

A New Model for OER: Anthologies Instead of Textbooks?

I have heard that sometimes adjunct faculty are assigned to teach a class at the last minute, often with the textbook already chosen and sometimes with a syllabus to follow. No one should argue that this is an ideal situation. But I also wonder how often instructors teach their courses to meet the align with the textbook they are using, rather than teaching the way that works best according to their context (e.g. students and their understanding of the subject)? I know I did it this way when I was a new faculty member. One of the claims regarding open source textbooks is that they lead faculty to think more deeply about their pedagogy and their contexts. This is what popped into my head when I read Cathy Saunder’s wonderful thought-provoking post: How Good Can a Course that Follows a Standard Textbook Be?

For some years, I’ve wondered why there tend not to be open textbooks in English Literature or Composition. OER publishers have told me that they prefer to publish open textbooks that have a large market. English faculty have told me they don’t typically use textbooks, but rather anthologies with commentary. Cathy made two points in her post that caught my interest:

“The class is, however, a huge one, with over 100 22-student sections taught each semester, and the instructor community is correspondingly large (even with many of us teaching 3 or 4 sections a semester).”

“Because 300-level composition courses with similar learning goals are rare, we don’t really fit into an existing market (though this doesn’t, of course, keep textbook reps from trying to convince us that they’ve got the perfect book for our class).”

These points made me think that there is indeed a large market for suitable English OER, and that we might be able to expand Cathy’s model of creating and sharing resources as follows. An OER publisher connects a group of interested instructors, who work together to develop not a textbook, nor an anthology, but a collection of readings, annotations, commentaries, and other resources. The OER publisher creates a platform that would allow instructors to easily create their own anthology, by picking and choosing from the collection of resources, which the platform would then turn into a finished book.  The book could be made available in a variety of digital or print formats for free or modest prices. One could imagine asking students to contribute to the collection as well in the form of renewable assignments that David Wiley has discussed and Robin DeRosa has experimented with. The platform would also keep the master copy of all finished books created, for use by those adjuncts who are hired at the last minute or anyone else who didn’t wish to create their own customized anthology.  What say you @OpenStax & @LumenLearning?

Posted in Uncategorized | Tagged | 1 Comment

Welcome to OER Week of OpenLearning’17

OER, which is an acronym for Open Educational Resources, seems like a no-brainer to me. In my 30+ year career teaching Principles of Economics, I have used a variety of books, including traditional commercial textbooks, books written on economic subjects for lay audiences (in lieu of a standard textbook), and no text book at all. I’ve concluded that most intro books are pretty much the same, and that any experienced instructor can teach effectively using pretty much any intro text. Why then should we ask our students to use commercial textbooks that cost $75, $175 or $375, when there are free or very inexpensive alternatives available? Yes it takes some time and effort to adopt and transition to using a new text. I don’t discount this, but as professionals don’t we have an obligation to do that if we think it would be better for our students?

Over the last few years, I’ve adopted OER for my intro courses. Before we go further, let’s get a few myths out of the way. When I talk about OER this week, I’m not referring to class materials, bits of content, what used to be called learning objects. I mean open source, complete printed or digital texts edited and published by organizations with solid reputations. These are not self-published books.

For faculty who don’t have experience with OER, there are five key questions we will explore this week:

1. What is OER?
2. How good are open source textbooks?
3. What do instructors and students who use OER think of them?
4. How do we find OER in our discipline?
5. What comes after adoption of OER?

Monday: What is OER?

Open educational resources are much more than simply free resources. In addition, OER give students and teachers the following five rights:

  1. Retain – the right to make, own, and control copies of the content (e.g., download, duplicate, store, and manage)
  2. Reuse – the right to use the content in a wide range of ways (e.g., in a class, in a study group, on a website, in a video)
  3. Revise – the right to adapt, adjust, modify, or alter the content itself (e.g., translate the content into another language)
  4. Remix – the right to combine the original or revised content with other material to create something new (e.g., incorporate the content into a mashup)
  5. Redistribute – the right to share copies of the original content, your revisions, or your remixes with others (e.g., give a copy of the content to a friend)

While you may not yet fully understand the “5 Rs,” as David Wiley calls them, but understand this: they dramatically change the role of a textbook in a course (or at least they offer the potential for that).

  • Read FAQ for Policymakers: Open Educational Resources https://docs.google.com/document/d/1px3jCbMW-bzhc7oNwxgaphTrurTr39lnowq1ZyE8Ec0/mobilebasic

And/or

  • What is OER? http://libguides.tcc.edu/faculty/OER/oer-info

And

  • David Wiley, “On the Relationship Between Free and Permissions in “Open” https://opencontent.org/blog/archives/4783

 

  • Join in – Open Dialog Session: The Open Textbook Library
    Are you concerned about higher education access and affordability? Open textbooks can help ease the burden of textbook costs for students and provide faculty with high quality, customizable course content. Join us for a discussion of textbook costs and student success.

4pm EDT; Tune in via Zoom & submit questions via Twitter.

Tuesday: How good are open source textbooks?

How can we be sure that the quality is equal to that of traditional commercial textbooks? Are open source textbooks peer reviewed? Do they go through a comparable quality control process?

  • Read How Good Can a Free, Open-Source Book Really Be? http://pedablogy.stevegreenlaw.org/2014/12/how-good-can-a-free-open-source-book-really-be/

And

  • How Good is a Free, Open-Source Text Part 2: http://pedablogy.stevegreenlaw.org/2014/12/how-good-is-a-free-open-source-text-part-2/

 

  • Join in – Conversation with John Hilton III about the research on OER efficacy. http://OpenEdGroup.org
    12 noon via Zoom conference

Wednesday: What do instructors and students who use OER think of them?

 

  • Join in – Twitter chat 2:00-3:00 pm

Thursday: How do we find OER in our discipline?

  • http://umw-oer-summit.economooc.com/how-to-find-open-textbooks/
  • http://libguides.tcc.edu/c.php?g=304199 OER libguide from TCC
  • Interview with Open Textbook author, Caitie Finlayson (recorded video) Twitter

 

  • Join in: Q & A with Caitie Finlayson, TIME: 11am-12noon EDT

Friday: What comes after adoption of OER?

Some have called open source textbooks as the gateway drug to something more bigger. As the week comes to an end, we can pursue two possible threads

1. Perhaps the best attribute of OER is not that it’s free, but that it makes instructors think profoundly about how they teach the course.

  • http://openlearninghub.net/week-eight-open-pedagogy/

2. Digital Courseware as a textbook alternative

  • Read Digital Courseware https://library.educause.edu/topics/teaching-and-learning/digital-courseware

And/or

  • Courseware http://postsecondary.gatesfoundation.org/areas-of-focus/solutions/digital-learning/courseware/

And/or

  • Waymaker: Personalized Learning Meets Open Education
    http://lumenlearning.com/courseware-waymaker/

Join in – Twitter conversation 2pm – 3pm

Posted in Uncategorized | Tagged | Leave a comment

Civic Engagement + Digital Literacy = Liberal Learning in the 21st Century

I’ve had a lot of thoughts swirling around in my head so far in 2017. This week, they are starting to come together. As I try to articulate them, you may wonder what this has to do with open learning, but please bear with me.

Dr. Troy Paino, our new president at the University of Mary Washington, gave a speech last month in which he presented his vision for our future. I found his remarks both traditional and progressive. He asked what UMW should become, given our history, culture and geographic location.

UMW is a public liberal arts and sciences institution. While we have colleges of business and education and a growing number of masters programs, liberal education is our core. UMW was originally created as a normal school, a women’s college to train teachers. Subsequently, it became the women’s college of the University of Virginia, and then a public liberal arts & sciences school in the mid-1970s. Located in Virginia, half way between Washington, DC and Richmond, UMW has had a rich background in American history, from colonial times when George Washington grew up nearby, to the American Revolution with Patrick Henry, George Mason, Thomas Jefferson and others who built this republic, to the civil rights movement in which former UMW faculty member James Farmer played an important part.

President Paino argued that UMW should build a future using a two-pronged approach: First, to create students who are civically engaged. Second, to create students who are digitally literate. These, he argued, are substantially related.

Students, Paino said, should understand the relevance of their liberal education to the problems of the world. But that’s just the start. Students need then to engage with the community, to make the world a better place, initially as part of their undergraduate studies, but then implicitly through their lives beyond university. This is what civic engagement means. A few years ago, I asked students if they had come to UMW because it was a liberal arts & sciences institution. Almost none of the students said yes. We need to do a better job of helping our students understand why that matters, and why fundamentally higher education at a school like Mary Washington is more than just completing one’s major.

Students also need to become digitally literate. UMW has a long and well recognized history in this area led by our Division of Teaching and Learning Technologies and encompassing UMW Blogs, which provided a blog server for faculty and students who wanted to engage with the digital world, our Online Learning Initiative, which sought to imagine what liberal learning could be in a blended or online course environment, to the Domain of One’s Own Project, which started at UMW, giving all students their own web domain and the resources to build those out in whatever way they chose. Despite a new major in communication and digital studies and a minor in digital studies, I think it’s fair to say that digital literacy does not currently play a central role in our curriculum, and most graduates do not take advantage of the opportunity to incorporate this into their degrees.

President Paino asked how can we adapt the liberal arts and sciences education provided at UMW to the digital world in which we now live. He observed that the world has changed from one where information was scarce and the goal of education was to provide that information, to one where information is abundant. In the latter, a key role for education has become to help people critically evaluate and filter the flood of information. How then, can we teach students to be critical thinkers in the world today? This is not a trivial question. This week, the Association of American Colleges & Universities released On Solid Ground, the initial results of a study of student achievement as measured by the AAC&U’s VALUE Rubrics. The study included results from 92 universities and looked at, not standardized test scores, but actual student work to evaluate learning in three areas: Written Communication, Critical Thinking and Quantitative Reasoning. The findings were encouraging, but also showed room for improvement. To quote from the press release:

“In the area of Critical Thinking, students demonstrate strength in explaining issues and presenting evidence related to the issues. However, students have greater difficulty in drawing conclusions or making sense out of or explaining the importance of the issue studied.”

Despite what the new U.S. Secretary of Education said yesterday, our job as faculty is to teach students not what to think, but how to think.

I may be the last person to realize this, but as I was participating in this week’s OpenLearning’17 activities, listening to Bryan Alexander’s Monday introduction to information literacy, reading the ACRL Framework for information literacy, and watching the NMC webinar on Digital Literacy, it occurred to me that in this age of alternative facts and fake news, there may be nothing more important than teaching citizenship and civic engagement through information and digital literacy. President Paino claimed as much in his speech, saying that beyond traditional notions of liberal education, “our fundamental purpose [today] is the preservation and advancement of our representative democracy.”

These are strong words. To make them real, we have much work to do. But if not us, who?

Posted in Uncategorized | Tagged | 2 Comments

Who is in the Open Learning ’17 Community?

Where is our community? The above google map shows the locations of everyone who was registered, at http://openlearninghub.net as of this morning. We will update the map over time so feel free to check back.

Posted in Uncategorized | Tagged | 1 Comment

Virginia’s Faculty Collaboratives Project

I am excited to work on the AAC&U’s Faculty Collaboratives project. The project is being implemented at the state level. Half a dozen states were in the first cohort, which began last year. Virginia is in the second cohort, which means we have benefited by learning from the experiences of the first cohort.

The nominal goal of the Faculty Collaboratives project is for each state group to develop a collection of faculty development resources to increase awareness of how to enhance the effectiveness of teaching and learning, especially in the liberal arts & sciences. These resources will be based on the findings of learning science over the last few decades–findings of which I suspect most rank & file faculty are unaware. I don’t know about you, but learning science was not part of my graduate program. AAC&U has conducted a number of initiatives in this area, including the two I’m most familiar with: LEAP and VALUE. The challenge is to get these resources into the consciousness of faculty.

Gardner Campbell, who is leading the Virginia initiative, expresses the challenge this way: How can we help students (and faculty) see general education requirements as more than checkboxes on the road to the student’s real education (e.g. in the major)? How can we help students and faculty see general education as more than just a set of hurdles to be gotten past? How can we help students and faculty see general education as a foundational part of their undergraduate studies as well as lifetime learning?  In short, how can we do what we’ve always valued, only better.

What approach are we taking in Virginia? We decided early on to use open educational practices as the model for our approach. Virginia has a vibrant open education community, so we have both resources and interest to draw from. David Wiley describes what we’re planning as rethinking general education requirements as if the Internet existed.

Open education can be pursued from a variety of angles, including open pedagogy, open access, open educational resources (OER), and more. There is no universally accepted definition of open education, which allows a diversity of approaches to be tried. It’s probably best to think of open education as a philosophy of teaching and learning, a set of practices, rather than a specific technique or set of skills. One can try some, but not all practices, and that’s fine. If this sounds imprecise, we will be exploring these concepts in more detail soon.

Our project will operate at several levels.

  • We want to create an online platform to showcase a set of teaching resources and practices.
  • We want to make the resource organic and sustainable so that the platform doesn’t die when the project is over.
  • We want to model what we’re presenting, using open educational practices.
  • We want to assess whether the platform makes a difference in faculty teaching practice and effectiveness.

Not too ambitious, huh? Beginning later this month (January 2017), we will conduct an open online course to explore open educational practices. The idea is to give interested faculty the opportunity to learn by doing. The course is designed for anyone who would like to learn more about open education, regardless of your background and experience. Everyone is welcome!

The course will be organized by topic. If you are only interested in one or two topics, you can simply participate those weeks.   Of course, you’ll get far more out of it if you participate in the whole course. If you have tried taking a MOOC (e.g. Coursera, Udacity) before, you will find our course very different. It will be based on the Connectivist MOOC model developed by George Siemans and Stephen Downs and exemplified by DS 106. For additional information on the course and how to participate, check out the project hub at http://openlearninghub.net. I hope to see you there.

Posted in Uncategorized | Tagged | 5 Comments

Letter to a New President

One of the unfortunate legacies of nearly two decades of UMW’s inconsistent leadership has been the creation of program silos moving in different directions. Many faculty like that. They like being able to do their own thing, and many of these things are worth doing, but that doesn’t mean these things are consistent with each other.  There is a widespread practice of people (faculty, departments, colleges, etc.) working to solve their own problems with no regard for how those solutions impact other programs.

Many of our faculty have never experienced strong leadership. By strong leadership, I mean leadership with a plan for our future, clearly articulated to faculty, who generally buy in to it. At present, there is little sense of institutional identity among faculty, who see themselves as economists, biologists and anthropologists but not as members of the university community. In part that is because we lack a clear sense of what the institution is or what it is trying to become. We had that sense some decades ago, but we’ve lost it. We are three colleges, but how they relate to each other is unclear, and the role of the arts & sciences in the whole is also unclear. The only message we (in the arts & sciences) have received in recent years is “keep doing what you’re doing”–which helps explain the silos.

We need senior leaders who convey hope, not hopelessness. I’m tired of hearing “Well, sure, but we can’t do that. We don’t have the budget.” If you believe something is important, FIND THE BUDGET. Otherwise, it’s not important.

Good leaders motivate/encourage their staff to do their best work, to work beyond what they are capable of on their own. By that measure, UMW has lacked good leadership.  As a faculty member, I haven’t had any substantive feedback about any of my professional duties from my department chair, dean or any leader in years. No one has said I’m doing a good job or a bad job in any aspect of my professional life.  One reason expressed has been “why evaluate if there’s no budget for pay increases?”  That seems shortsighted at best. See above re: strong leadership.  Formal evaluation is secondary.  It’s communication and feedback which are of primary importance.

In recent years, I have regularly heard faculty across the disciplines/colleges say things like:

I’m not responsible for admissions, retention, completion, advising or anything outside my unit, or even some things within.

The term for these professional activities is “appreciated, but not valued”. The administration wants faculty to do these things; indeed, they are necessary for the success of the institution. Doing these things well can take significant thought, effort, and time. But faculty do not receive any significant credit or reward for doing so. These things are “counted” towards salary increases and promotion in the same way as attendance at department meetings is counted.  As a consequence, lacking a strong sense of identity as a member of the institutional community, many faculty simply go through the motions.

Faculty feel underappreciated and exploited.  I expect staff feel that way too.  Appreciation doesn’t require salary increases; indeed, salary increases alone are probably insufficient. Why has there been no public recognition of excellent teachers, excellent scholars, extraordinary service?

There are many faculty who are interested in making the institution better, making things right. They have been tolerated, but ignored. They have not been given the authority or support to make (much) positive change.  Many are discouraged, but most are still open to the possibility of something better. You could start with them.

 

Posted in Uncategorized | 1 Comment

Course Evaluations for my Waymaker Course Pilot

LumenWaymaker-400x80This past term, as part of a nation-wide pilot including more than 9000 students, I pilot-tested Lumen Learning’s Waymaker platform in the online version of my ECON 202, Principles of Microeconomics course.  I hoped that Waymaker would provide a useful structure for my online course.  At the end of the term, I asked the students to complete a course evaluation that I constructed. The responses were anonymous, though I gave each student who submitted one extra credit towards their final grade. (I ask them to email their responses as anonymous attachments to my colleague, Bob Rycroft, who sends me the list of names, but holds the evaluations until final grades are turned in.) Out of 36 students in the course, I received 28 submissions so the response rate was decent.

The evaluation included questions on the online course (since I’m still trying to improve my online teaching) as well as on the Waymaker platform. The questions asked for student perceptions and included both quantitative and open-ended responses. My statistical analysis of learning outcomes for this course, as measured by student performance, will come later.

This past week I reviewed the student submissions and coded them into Excel. I prefer to do this myself, rather than asking a student aide to do so, because in reviewing and coding I find myself doing some processing of the results. More precisely, because I’m working one submission at a time I get a sense for what and how each student is thinking; I get a sense of consistency or inconsistency in responses to different questions, and I get a sense of the relationships between responses on one question and responses on another.   For example, and this is just a casual feeling, it seemed like non-traditional age students were less likely to buy into the Waymaker approach, which attempts to facilitate deep learning. This manifested in things like not watching the video content, not doing the simulations, and asking for study guides that would explain exactly what they were expected to learn. Don’t get me wrong; I am not anti non-traditional age students. Some of my best students fit that demographic.

So here are my initial impressions.

ECON 202, Principles of Microeconomics is an introductory-level course. The course is required for the business, environmental science and international affairs programs, as well as for certifying to teach K-12. While the intent is for students to take this course early in one’s academic career, some defer it until later. The majority of my respondents were in their first (10 students) and second year (11 students). The remaining seven students were third (5 students) and fourth years (2 students). Most of the students (18) live on campus. Three live close by; the remaining seven live as far as 50 miles away.

Interpreting a survey like this is challenging. Some students seemed to try to say what they thought I wanted to hear. Some gave answers to related questions that were inconsistent with one another. Some students didn’t interpret the questions as I intended or didn’t think carefully about them, dashing off answers that don’t provide much useful information. One student wrote,

“The hardest material to learn was anything really extensive and difficult.”

An additional complication was that this was an online course, which is relatively rare at my institution, so some of the criticisms expressed by students were criticisms of online courses, rather than of the Waymaker platform. Similarly, economics is a challenging subject, since unlike English or History, many students haven’t studied it before. So some of the criticisms were really about the discipline rather than Waymaker.

Waymaker is a next generation courseware environment, combining text, video, and animation/simulation, all embedded in a mastery learning framework with formative and summative assessments interleaved through each content module (analogous to a book chapter).

Three quarters of respondents were generally favorable towards the Waymaker modules, and only 18% said they preferred a traditional textbook. 42% of respondents said Waymaker was easier to understand than a traditional textbook, while 32% said a commercial textbook was easier. As one student said,

“The combination of videos, self-checks [formative quizzes], and worked examples allowed for small checks for understanding that a textbook does not provide.”

Students indicated that the hardest material to learn was specific economic theories, understanding graphs or understanding the math. Anyone who has taught principles of economics would find this pretty normal, but I wonder if Waymaker could do better than text, given its multimedia approach?

Let’s look at some of the specific features of Waymaker.

 Videos:

None Some All  
1 20 7 How many videos did you watch?
3 13 12 Did you watch them in their entirety?
No Yes
12 16 Did you ever watch the same video more than once?
Not very useful 1 2 3 4 5 Very useful
0 6 6 8 4 On a scale of 1 to 5, how useful were the
25% 25% 33% 17% videos for learning concepts or models?

Only 25% of students (7) indicated that they watched all the videos. Is this a problem? Slightly less than half the students watched the videos in their entirety. More than 60% (16 students) watched the same video more than once, while 75% found them somewhat to very useful.  25% indicated the videos were not very useful.

Animations/Simulations

None Some All  
1 18 9 How many animations/simulations did you play?
2 15 11 Did you play them in their entirety?
No Yes
10 18 Did you ever play the same animation/simulation more than once?
Not very useful 1 2 3 4 5 Very useful
1 6 10 5 4 On a scale of 1 to 5, how useful were the
4% 23% 38% 19% 15% animations/simulations for learning concepts or models?

About a third of the students played all the animations/simulations, and the same number found them somewhat or very useful.  27% found them not very useful.   My impression is that students who like them, liked them a lot.  I didn’t get the same impression with the videos.

Study Groups:

Less Same More
3 10 15 Did you learn more from working with your study group than you would have learned on your own?
Own Group When you did group assignments, did you mostly work on your own & share, 
7 21 or did you work together as a group and go over each others’ contributions? 

One of the features of the online course was study groups, which I used to counter possible feelings of isolation for students working online. Slightly more than half the students (54%) said they learned more from working with their study group than they would have learned on their own. About 11% said they learned less. Three quarters indicated that they worked together as a group on assignments, while one quarter divided up the assignments, worked individually, and then shared answers. Some study groups were effective, because the student drew on each other’s strengths. Others were less effective, either because one or more students didn’t participate easily (or were hard to reach), or because “no one in the group knew the answers.” I need to do more research on group work to see if I can improve these results.

Metacognition

No Scores Yes  
1 8 17 Did you use the self-checks to assess your learning? i.e. Did you use the self-checks to think about what you knew or didn’t know, or did you just focus on the scores.
Never Some Often  
4 11 13 Did you use the module quizzes to study for exams, aside from doing them for the grade? 
Less Same More  
7 10 11 Compared to using a traditional textbook, did you think about how well you were learning as you went through the modules?

Nearly all students used the self-checks (formative quizzes); only 28% (8) focused on the scores, while a majority (17 students – 68%) used them metacognitively to think about which topics they had mastered and which they had not. Nearly half (13 students – 46%) used the summative end-of-module quizzes often to study for exams, while another 39% (11 students) used them sometimes. 39% (11 students) indicated that thought more about how well they were learning using Waymaker compared to a traditional textbook. 25% (7 students) said they thought less about how well they were learning using Waymaker.

Less Same More  
5 17 6 How much time did you put into this course compared to others you have taken at UMW? 
Easier Same Harder  
0 7 16 How did this course compare with the others you have taken at UMW?

A majority of students (57%) found the course harder than others at UMW, but only 26% put more time into the course, which seems odd to me.

What tentative conclusions have I drawn from these survey responses?

Some students liked Waymaker; some students didn’t. What was the difference? Was it merely a question of taste, or something more substantive that could be improved?

Weaker students seemed to not like Waymaker, but was this cause or effect?

Some students didn’t seem to take the Waymaker features very seriously or they didn’t give them a serious try. For example, a number of students didn’t do much with the videos or the animations/simulations. Look back at the Video & Animation/Simulation results above. Students who responded None to the video or animation/simulation questions obviously didn’t benefit from them, but I suspect that many of those who responded Some/Some/No were also disengaged.

At the same time, 50% of the respondents said that the videos were useful or very useful for learning concepts or models, while only 25% said they were not useful. Similarly, 35% said the animations/simulations were useful or very useful, while 27% said they were not.

My impression is that some students, especially first semester first year, or transfer students, don’t seem to understand learning at the university level. Some don’t know how to study deeply and effectively. They have adopted the “school is about grades” motivation, and their approach to studying is to skim the readings looking for key terms to memorize. I know not all FY students or transfers are like this, but it’s a pattern I saw in the survey.

Waymaker was designed for deep learning; it was designed to mimic Socratic questioning of the student. It’s not perfect yet, but those were some of the goals.

Two common themes in the student comments were time management/self-discipline and the greater time required compared to a face-to-face course. The first theme is common to online courses. As one student explained it, “Being responsible and getting assignments done without having the professor in class reminding you.“

The second theme was more interesting. A number of students objected to the time required to work through the modules. But compared to what? The time spent reading a traditional text? But that’s not the right comparison. Waymaker tries to provide the structure students get in a face-to-face course. So the proper comparison should be time in the classroom plus time spent reading the text. I don’t think most students understood that.

Several students mentioned that the course required them to teach themselves! But what do they mean by that, and how is it different in a face-to-face course? If they simply mean the lack of lectures meant they had to filter the content and discern meaning for themselves, that’s almost tautological. And some research suggests that might actually enhance learning, for those that do it seriously.

But what if part of the pushback was students’ lack to experience with studying in depth? Then if they had bought into the program they could be more successful, but if they didn’t buy in they could be frustrated and unable to see the superficiality of their study approach, possibly leading to giving up on Waymaker. We might be able to test this if we have time-on-task data or some other measure of engagement.

I’m ending up with at least as many questions as answers here. Much to ponder. Next fall, when I teach both face-to-face and online sections using Waymaker in both, perhaps I’ll be able to tease out some more answers.

 

Posted in Uncategorized | 4 Comments

Response to Robin DeRosa’s “My Open Textbook Pedagogy & Practice”

This is an amazing and thought-provoking post. There’s so much here that I just want to put my initial thoughts out; otherwise, I may put it aside and not respond at all.

I’ve done this sort of thing before but never as ambitious a project as a complete text—see http://2008financialcrisis.umwblogs.org and more recently http://WhitherTheEuro.umwblogs.org —so I can speak to how much work and energy on your part is involved. I can also affirm that students who participate engage with the project far more deeply than with most academic assignments.

First, the naysayer: Did you review and/or edit the students’ commentary? That could move it from an excellent student project to a more serious professional resource, albeit at the cost of taking something away from the students, and more of your time. For my financialcrisis project, I ended up spending a huge amount of my summer reviewing, editing and other production work. Do you have a better way?

In my view, there are a couple of takeaways here:

  • Don’t let the perfect be the enemy of the good. One shouldn’t compare an open text like this one to a finished, glossy commercial text. A better comparison is with a first draft manuscript—the one the author submits. The content is pretty much all there, though the finish may not be. But that’s okay.
  • It’s hard to understate the learning experience of the students. This learning experience is so much more profound than what a student normally gets from reading a book, because the process of curation requires students to examine assumptions, consider alternatives and evaluate what they conclude—all things that we at least subconsciously hope students will do when they read, but which realistically very few do—in part because we haven’t taught them to read that way, and in part because it’s hard work and takes a lot of time.

I look forward to reading the next chapter in your exploration of open texts.

PS: As I was thinking about responding to Robin’s post it occurred to me that the way I respond to a blog post is a little different from the way I respond to a colleague’s paper. Perhaps it’s the nature of blog posts, which are less formal than papers. Whatever the reason, I find I respond affectively to blog posts as well as cognitively.  It’s not that a post can make me feel good or bad, but rather my response is coming from a deeper level than when I respond to a formal paper.

 

Posted in Uncategorized | 1 Comment