Dr. Adam Moser: Research and Teaching

Critical thinking is thinking that questions, if you believe wikipedia.  This sounds like questioning assumptions, which seems like part of critical thinking.  Not what did someone just say, but more why did they say it and why do I believe them (or not).  I’d also add that I roll up problem solving into critical thinking.  I am presented with a problem and I must determine a solution from information (from my head, provided in the problem, or out in the world I must get).  After I find a solution I need to reflect if it makes sense.  I feel like critical thinking is a skill I want my students to have and I’m pretty sure every college mission statement includes it.  I also have had the experience it seems difficult to teach.  I come up with more advanced problems that require multiple concepts, more open ended questions that don’t have an obvious answer, or ask them to think about why we are working on a particular topic.  When I do students usually run the other direction (e.g. equations & calculators, ask for the answer, etc).

A good friend of mine at the Center for Excellence in Teaching and Learning at the University of Rochester is working with undergraduate students helping them teach organic chemistry.  Here was a comment one of the workshop leaders made.

“I think the problem with getting students to think critically in Organic Chemistry isn’t solely an issue of them not being interested in the material. I think there’s a much larger, underlying problem… I think that often times, when taking a science class, there is often a feeling of “why are these questions being asked?” There’s a feeling of apprehension in trying to think critically, or in other words, creatively, about a subject that we already perceive as “solved.” The lack of critical thinking, it seems to me, is due to the fact that students perceive the science that they’re being taught as being “right” and that there is no reason for them to give it more than the cursory attention necessary to learn that which is already known. They have no motivation to think critically because there’s really no problem to solve; all that these questions are asking them is do they remember how someone else solved this problem.”

I’m not sure students are thinking this deeply about why they aren’t motivated and engaged, but even if this isn’t the conscious (or even subconscious) reason, trying to attack this problem might yield dividends.  I certainly would have put “not interested in subject” or “has no reason to care” as the top reason students don’t engage in critical thinking.  But maybe it is “not interested in already solved problems”.  If they know there is already an answer out there, why fight through it?  Why work through the problem, when the answer is usually so available (its in the professors head who is standing right beside me).

My first response is that when they leave campus, their job will likely be mostly problems that are not solved or the answers are not easily obtained.  Jobs that only rely on information that can be looked up or operations that are so mundane they don’t require critical thinking are likely to be replaced by robots and computers or shipped off to a country with a lower wage.  There is a strong likelihood they will be working a job that requires new/more information that were taught in college.  Basically, critical thinking is the MOST important skill they can get.

So now I start questing if I’m doing my job well.  They need background (information, definitions, context), skills (algorithms, procedures), and concepts (big ideas, why this matters) before they have a good chance of being good critical thinkers, but am I moving them in that direction fast enough?  Is the department’s curriculum getting them there before they graduate?

UPDATE (1/4/2013) – Great article about how to teach critical thinking and its attributes.

In “Deep in the Heart of Texas“, Stanley Fish notes that we aren’t always in the best position to judge the quality of something.  We don’t always know immediately after we obtain something what its actual worth is.  This is true he says for education, and why he argues against student evaluations as a measure of teaching quality.  Instead we need “Deferred Judgment” to evaluate teaching because we don’t know how useful the education was right now and it might be a long while before we do know.

And that is why student evaluations are all wrong as a way of assessing teaching performance: they measure present satisfaction in relation to a set of expectations that may have little to do with the deep efficacy of learning. Students tend to like everything neatly laid out; they want to know exactly where they are; they don’t welcome the introduction of multiple perspectives, especially when no master perspective reconciles them; they want the answers.

But sometimes (although not always) effective teaching involves the deliberate inducing of confusion, the withholding of clarity, the refusal to provide answers; sometimes a class or an entire semester is spent being taken down various garden paths leading to dead ends that require inquiry to begin all over again, with the same discombobulating result; sometimes your expectations have been systematically disappointed. And sometimes that disappointment, while extremely annoying at the moment, is the sign that you’ve just been the beneficiary of a great course, although you may not realize it for decades.

This got me thinking because while I agree that deferred judgment makes more sense, I would like to know my impact on the students now.  I need to know a.s.a.p. because I need to make changes.  Additionally, I don’t think we track students well enough to get that deferred judgment perspective.  How often do we bring back 5 year alums to discuss with the faculty what worked and what didn’t (as opposed to alumni reunions that at all about fun and fund raising). This idea also scares the pants off a lot of faculty whose jobs are dependent on strong student reviews.  Hopefully my dean and chair have read this article.

And what about the students?  Do they understand that their work in the course might be paying dividends later, but just be confusing and a pain now?  I don’t ever remember anyone helping me understand that when I was a student.  No one explained that a lot of the courses I was talking were about learning to think about ideas just as much as they were about the specific ideas.

I have no conclusion or witty fix for this.  Having a discussion with the students about it sounds like a step in the right direction, but if I used my class time to chat with the students about all the nuances of education I think they should know, then we would never actually get to any of the discipline.

UPDATE – (12/16/2012) “Evaluate the Evaluation: Course Evaluations and External Biases” discusses biases in course evaluations.  More evidence it is tough to really get the impact of your teaching.

I’m not sure how it happened, but I ended up having two Washington Post articles up on my screen at the same time: “Is college too easy? As study time falls, debate rises.” and “I went to some of D.C.’s better schools. I was still unprepared for college.“  One article reports college is too easy, while the other is a student basically showing how college was very challenging because he was so unprepared despite going to a “good” high school.  So which is it?  So now the Goldilocks question: Is college too easy, too hard, or just right?

The article stating college is too easy cites the study on how the number of hours students study is on the decline (find data here) while the average grade is increasing (I’ve mentioned this before).  They even quote some students saying their high schools were more challenging.  In the other article a student goes step by step through his education and explains how it ended up that he was out of his league on the first day of college.  It is actually a really nice explanation of how a student can evolve and be successful.

I’m also not sure what “too hard” or “too easy” means.  Should college be a struggle at all?  The Times recently had an article titled, “Why Flounding Is Good“.  When we struggle we learn things more deeply and retain them longer.  I feel like when I struggle I appreciate what I learn more.  So maybe this is what “tough” really means.  How much should the students struggle?

So maybe a better question than asking how tough college is right now would be asking how tough college should be.  Almost every mission statement for college undergrads includes thinking critically, which I would say is by its nature very hard.  We want students to bring together information from different sources and synthesis ideas to answer problems.  We want them to form new ideas.  Now add on top if these all the skills, knowledge, and concepts necessary for a discipline.  Sounds challenging for everyone.  That seems like a lot for the students to develop in a few years.  From a faculty standpoint it is challenging generating a curriculum that would support each student in this development.  From the college standpoint, it seems impossible to assess well.  Add on the obvious problem that we are teaching diverse groups of students who learn in different ways, at different speeds, with different levels of motivation and interest.

I feel like I’ve moved away from the question of how tough college should be, too just what college should be.  I suppose once I feel like colleges have what they should be down and how to provide it better, I’ll come back too if we are being too tough on the students.

“The path to smart goes through dumb.”  A colleague of mine, Dr. Dan Dill, says this.  I really like the sentiment.  Students don’t realize that most of what they will learn in college won’t come easy and it will take hard work, but they really can do it.  They just need to fail faster and better.

The path to smart goes through dumb. ~Dr. Dan Dill (Boston University)

Chronicles of Higher Education recently published “Next Time, Fail Better“, which argued that humanities majors need to learn the lesson of failure much as computer science students are forced to.  And Inside Higher Ed published the importance of letting students fail.  No one likes failure.  Worse many students come to college never having really experienced it.  Most come in believing what worked in high school will work in college, and when it doesn’t they don’t know how to adapt.

Recently, I had a student in my office at the end of the semester who had done poorly on every exam.  I asked her how she studied at the start of the semester and she described reading the book and looking over the solutions to problems done in class.  I asked her after she did poorly on her first exam what did she change.  She said she just tried to study hard (i.e. more of the same).  I asked her after her second bad exam what did she do?  Study harder!  She had no idea how to deal with failure.  She had no concept of alternative ways to study.  Even worse she didn’t even think to ask a staff member.  I kept thinking that was exactly what I’m here for, to help you find new ways to learn.  This student falls perfectly into Einstein’s definition of insanity, “Doing the same thing over and over again expecting a different result.”

I’m not at all shocked students don’t have the metacognition to realize in advance what they are doing isn’t working.  I’m not shocked they don’t know alternatives to the study/work habits they learned empirically in high school.  How could they when no one has shown them how?  I am shocked that they just keep going.  To take this a bit further, I need the students to fail for me.  That is how I learn what they don’t know and can help them fix it.  I often tell students,

“I have not yet become a good enough teacher to look deep in your eyes and see the problem.  I need you to try and make a mistake (i.e. fail) so I can help you.”

College is a great time to make mistakes.  You don’t get fired, you get help.  As long as you try and fail early enough to get it fixed, you are on your way to smart.

The New York times always has some interesting comments on education.  “Vocation or Exploration? Pondering the Purpose of College” discusses what is the point of college.  Students come to college for all kinds of reasons (or maybe hardly an reason at all).  Think made me go look for some data about students coming to college.  I found the following quotation and data at National Center for Education Statistics.

“Enrollment in degree-granting postsecondary institutions increased by 9 percent between 1989 and 1999. Between 1999 and 2009, enrollment increased 38 percent, from 14.8 million to 20.4 million. Much of the growth between 1999 and 2009 was in full-time enrollment; the number of full-time students rose 45 percent, while the number of part-time students rose 28 percent.”

More students are going to college, with growth faster than just standard population increases.  All these “new” students that are attending that weren’t in the 1970’s must be there for a reason.  I know so many students that go to college with little idea of what they want to do or what really motivates them, but it seems like they are coming anyway.  So unless you want to be a vocational school maybe the goal needs to be giving the students experiences where they find what they will be happy getting up every morning to do.

UPDATE (5/31/2012) – Just to make this more interesting a study that I’ve seen sighted lots of places argues colleges don’t teach the students much anyway.  They test for critical thinking skills and ability to make logical connections.  This study is the Collegiate Learning Assessment (CLA, http://www.collegiatelearningassessment.org/), but you might go read the book Academically Adrift, which puts a lot of information together.

UPDATE (9/12/2012) – Chronicles and Higher Ed both reported on a study that finds even though a high education degree doesn’t shield you from unemployment, it does help.

UPDATE (1/10/2013) – Chronicles discusses how recession hurts college graduates, but less educated works are being hurt even more.  Even if this recession is hurting college graduates more than previous down turns, it seems a degree still helps.

UPDATE (5/20/2013) – It isn’t just the degree, it is what the degree is in when it comes to employment outlook.  This isn’t a shock I think.

Online education isn’t new, but recently its gotten a bit bigger.  Stanford taught 160,000 students CS221: Introduction to Artificial Intelligence.  MIT was already in the mix with MITx.  Now Harvard and MIT just put together 60-million dollars to work on a whole platform called edX.  Not to mention the non-University competition.  Obviously there is a lot of interest in providing education more broadly and outside the classroom.

Sebastian Thrun (Stanford)

The question in my mind is how will it be better or worse. The good seems obvious.  Online education can be made available to more students in more places at more times.  The student has more control over the speed and repetition of their input.  Assuming some thought is put into the process, I imagine that both the ability to ask questions, have a dialogue, and assess the students can all be achieved.  All the above assumes some slick technology, thoughtful faculty, and some disciplined students.  Then again that isn’t too far from what a physical classroom takes to be productive.

That seems to leave the real difference between the online and offline is the human contact.  Is there a difference in watching a video of me giving a lecture than seeing me in person?  Is there a difference in instant messaging/skype/google+ and a traditional office hour?  I think the answer is yes there is a difference, but maybe not the dramatic difference faculty might like to believe.  In a recent editorials on Chronicles of Higher Education (part I and part II), Jeff Selingo interviewed students at different institutions and one of the students’ comments was how face to face is more important than ever, given our digital social experience.  I know people have studied how strong emotion ties information to our long term memory.  That is why you remember what the entertaining, engaging, and motivated professor taught you longer than the boring professor.  Will the online counterpart be able to create this environment?

I don’t know the answers and I can’t see the future well, but it seems likely this isn’t going away.  It means there are changes coming for those traditional faculty/lecturers because unless they are bringing something very worthwhile to their students, they will likely be replaced by a more clear video, good education technology, and some solid tech support.

UPDATE: (5/15/2012) Purdue gets in on the action too.

UPDATE (5/31/2012) “The Future of Undergraduate Teaching” is a view of how undergraduate teaching may look in a world of online education.  Online lecturers with support staff and peer-to-peer learning in early coursework.  Live lectures are a rarity, while small group learning becomes the norm.

UPDATE (6/1/2012) Chronicles of Higher Ed just had an article summarizing a study saying hybrid online/traditional courses teach just as much.  The study was done by a consulting firm on technology in education, so take it was grain of salt.

UPDATE (6/19/2012) How about online education without teachers?  Chronicles had an article today on Peer-2-Peer University.  Here is the quote from the front page.

At P2PU, people work together to learn a particular topic by completing tasks, assessing individual and group work, and providing constructive feedback.

UPDATE (7/12/2012) “What the matter with MOOCs?” discusses the differences in online education compared to in class.

The delivery of course content is not the same as education. And training students to perform technical tasks, such as doing basic equations in calculus, is not the same as education.

Jon Chui is probably not the first, but has a very beautiful way to show off science.  He creates pictorial guides to science topics.  Below is my favorite one about the confusion with intermolecular forces.  Just click on the picture to take you to the larger image.  I wish there were more illustrations like this to teach from.

I have students all the time telling me how much they are studying.  Sometimes it pays off and sometimes it doesn’t.  I think a lot of this has to do with how people study.  Most students have never been taught how the learning process works and how to learn effectively.  Worse still, students don’t realize that learning is a biological process (neurons grow).  I’m not shocked that their first attempts to learn a topic aren’t successful.  Luckily, cognitive psychologists have been working on learning theory for a while and here are 3 examples of

Samford University has 5 cool videos on the following topics: Beliefs That Make You Fail…Or Succeed, What Students Should Understand About How People Learn, Cognitive Principles for Optimizing Learning, Putting the Principles for Optimizing Learning into Practice, I Blew the Exam, Now What?.  Each video goes through study skills, common missteps for learning, and learning theory.  These are great videos for new college students.  There are so many good points in these videos I can’t summarize them all here.

The Education Resource Center at Boston University a great (though a bit long, 50min) video (below) that speaks very specifically to how learning works.  It begins with a 20min explanation by Dr. Michael Grant about learning theory.  This is followed by 30min of study plans that are consistent with the learning theory.  There are a lot of tactics to studying here (some more useful than others).

Finally I want to point on an Article from Chronicles of Higher Education titled “Metacognition and Student Learning” by James Lang.  I think a big problem in student learning is that students don’t think about the learning process; they are just focused on getting the information in their heads to pass the test.  Most students trick themselves into thinking they really understand material because they saw the professor work the problem on the board, or that they understand the material because they memorized on the key terms and equations.  Metacognition is the act of thinking about your learning and learning process.  This can be tough to do (similar to editing your own work) because it is difficult to step outside yourself.  Here is a quotation by Dr. Chew from the article.

“Poor metacognition is a big part of incompetence,” he explained. “People who are incompetent typically do not realize how incompetent they are. People who aren’t funny at all think they are hilarious. People who are bad drivers think they are especially good. You don’t want to fly on a plane with a pilot who has poor metacognition. A lot of reality shows like American Idol highlight people with poor metacognition for entertainment. Everyone knows people who are seldom in doubt but often wrong.”

UPDATE (5/31/2012) – “Forget What You Know About Good Study Habits” in the NY Times has a nice summary of some misconceptions about studying.  This includes learning styles, teaching styles, study habits, how bad study habits can hurt, and a lot of studies to support the ideas.

UPDATE (12/16/2012) – “High School Daze: The Perils Of Sacrificing Sleep For Late-Night Studying” on NPR summarized a study that makes clear the importance of sleep to learning.  I’m not sure if anyone needs to be convinced that sleep is important for learning of real long-term value, but this helps explain the reasons.

UPDATE (1/29/2014) – Faculty Focus provided a little summary of Chew’s work.  Again emphasizing the misconceptions students (and faculty?) have about learning.

Let me start by saying this post is my attempt at bringing together a handful of ideas and information I’ve heard recently all having to do with technology changing the classroom.

First lets start with the qualitative plot to the left.  This was part of a Forbes article about Apple getting involved in education.  I don’t feel like I have to make much of a case that information is getting cheaper and education is getting more expensive (see college tuition).  Not only is information getting cheaper and easier to find, but the technology that feeds it to us is cheaper as well.  Almost ever moment of our lives are different because information is found and passed around so easily.  With that broad, mostly unformed thought in my head, I ran into the following.

I’m a fan of the Freakonomics books and the podcast.  I was listening to the podcast “How Is a Bad Radio Station Like Our Public-School System?“.  The full transcript is provided, but I’d suggest just listening to it (its free and ~30min).  One of the points they make is that if you fell asleep 150 years ago and woke up today, almost every aspect of life would seem very different.  One thing that isn’t that much different is most classrooms.  25 kids, 1 room, 1 teacher, one style of teaching going on (most likely a “lecture”).  One idea I took away from it, is if Pandora can customize your song list, why can’t we customize student learning.  With all the technology and information at our  disposal, why can’t we give the student exactly what they need?  Well there are people out their trying to do that.

The School of One idea provides many modes of learning to the student: Large live instruction, small live instruction, virtual live instruction independent practice, small group collaboration and independent virtual instruction.  This is like the synthesis of ALEKS, Khan Academy, traditional lecture, office hours, and peer learning all at once.  Even better they track how the students are doing and if the student seems to learn worse in one mode than another, they stop putting the student in the mode that doesn’t help (i.e. saying you don’t like a song in Pandora).

Now the classroom 25 students, 1 teacher, and 1 room.  The School of One classroom is whatever it needs to be for all the students.  Without technology I don’t think this idea is remotely possible, but with technology it seems like this is something that could be managed.  YouTube videos of lectures the students can watch, rewind, and skip through.  Online homework/tutoring to support and access their learning.  Small group work to give them a chance to work out their ideas.  Larger univocal lectures to expand on concepts.  In the picture you see lots of teachers, but you also see many more students because many students are working by themselves or with peers.

Everything I’ve linked above has to do with K-12 education, but it did get me thinking about what this means at the college level.  If all a professor does is stand in the front of a large room and talk at students, we should probably just record them (assuming the talking is any good) then just provide the video online.  Is it the professors job to provide various modes of learning to our students?  Unlike K-12 where most classrooms are ~25 students, many college classrooms are much larger, does that imply a college classroom on average would require more diversity in learning modes?  Why can’t we give our students exactly the kind of instruction that makes the most impact?

NPR had a great summary of how physicists have been gathering data on how ineffective lecturing can be and how that time can be used more effectively:  Physicists Seek To Lose The Lecture As Teaching Tool.  The link has a 8 min audio clip as well as the complete text of the article.

I don’t want to summarize it all because they explain it well, but I do want to quote the bottom.

Some faculty are threatened by this, but Mazur says they don’t have to be. Instead, they need to realize that their role has changed.

“It used to be just be the ‘sage on the stage,’ the source of knowledge and information,” he says. “We now know that it’s not good enough to have a source of information.”

Mazur sees himself now as the “guide on the side” – a kind of coach, working to help students understand all the knowledge and information that they have at their fingertips. Mazur says this new role is a more important one.