But did they learn anything?

The course transformations I work on through the Carl Wieman Science Education Initiative (CWSEI) in Physics and Astronomy at UBC are based on a 3-pillared approach:

  1. figure out what students should learn (by writing learning goals)
  2. teach those concepts with research-based instructional strategies
  3. assess if they learned 1. via 2.

Now that we’ve reached the end of the term, I’m working on Step 3. I’m mimicking the assessment described by Prather, Rudolf, Brissenden and Schlingman, “A national study assessing the teaching and learning of introductory astronomy. Part I. The effect of interactive instruction,” Am. J. Phys. 77(4), 320-330 (2009) [link to PDF].  They looked for a relationship between the normalized learning gain on a particular assessment tool, the Light and Spectroscopy Concept Inventory [PDF], and the fraction of class time spent on interactive, learner-centered activities. They collected data from 52 classes at 31 institutions across the U.S.

The result is not a clear, more interaction = higher learning gain, as one might naively expect.  It’s a bit more subtle:

Learning gain on the LSCI and Interactive Assessment Score, essentially the fraction of class time spent on interactive instruction. Each point represents one class with at least 25 students. (Prather et al, 2009) Our UBC result from the Sep-Dec 2010 term is shown in green.

The key finding is this: In order to get learning gains above 0.30 (which means that over the course of the term, the students learn 30% of the material they didn’t know coming in) — and 0.30 is not a bad target — classes must be at least 0.25 or 25% interactive.  In other words, if your class is less than 25% interactive, you are unlikely to get learning gains (yes, as measured by this particular tool) above 30%.

Notice it does not say that highly interactive classes guarantee learning — there are plenty of highly-interactive classes with low learning gain.

Back in September, I started recording how much time we spent on interactive instruction in our course, ASTR 311. Between think-pair-share clicker questions, Lecture-tutorial worksheets and other types of worksheets, we spent about 35% of total class time on interactive activities.

We ran the LSCI as a pre-test in early September, long before we’d talked about light and spectroscopy, and again as a post-test at the end of October, after the students had seen the material in class and in a 1-hour hand-on spectroscopy lab. The learning gain across 94 matched pairs of tests (that is, using the pre- and post-test scores only for students who wrote both tests) came out to 0.42. Together, these statistics put our class nicely in the upper end of the study. They certainly support the 0.30/25% result.


Okay, so they learned something.  How come?

The next step is to compare student performance before and after this term’s course transformation. We don’t have LSCI data from previous years, but we do have old exams. On this term’s final exam,  we purposely re-used a number of questions from the pre-transformation exam. I just need to collect some data – which means re-marking last year’s final exam using this year’s marking scheme. Ugh. That’ s the subject of a future post…

Clicker questions should be integrated, not jammed in later

The CWSEI group at UBC gets together every week to discuss a journal article. This week, it was a new article by Melissa Dancy and Charles Henderson “Pedagogical practices and instructional change of physics faculty,” Am. J. Phys. 78 (2010).

One of the questions explored in the paper is, why don’t physics faculty members adopt the research-based instructional strategies that so many have already heard of? Mazur-style peer instruction (PI) using clickers, for example.

Dancy & Henderson discovered that nearly two-thirds (64%) of the 722 faculty who completed their survey were familiar with PI and 29% actually used it in their classes. But on further probing, it turned out only 27% of that 29% (we’re down to about 8% now) had students discussing ideas and solving problems multiple times per class. It appears that a lot of physics faculty members equate “peer instruction” with “yeah, I’ve got clickers in my class.” The technology is there but it’s not being implemented in a way that promotes learning. Continue reading

Using clickers to engage your audience

Today, I had the pleasure of demonstrating clickers to a group of journalists visiting UBC. They’d heard of the Carl Wieman Science Education Initiative that I’m part of, and wanted to “see clickers”.

The message I gave them is the one Tim Slater (@caperteam), Ed Prather and the rest of the team at the Center for Astronomy Education have been preaching for years: it’s not what the instructor does that matters; it’s what the students do for themselves. And clickers are a tool for facilitating this learner-centered instruction.

One thing a clicker question can do is quickly engage the audience. These journos are from France, so I posed a simple question

How tall is the Eiffel Tower?

A) 162 metres
B) 324 metres
C) 1024 metres
D) none of these

Is it important students know the height of the Eiffel Tower? No. But it engages them, gets their attention. And the crazy things is, after answering this question, you really want to know!

My audience didn’t all chose the same answer and I didn’t tell them the height. We moved on to other kinds of clicker questions: gathering predictions and think-pair-share (aka peer instruction).

But evidently the Eiffel Tower question was smoldering in a few of them because at then end of my presentation, they asked me to go back to the Eiffel Tower question, and proceeded to have a debate about its height. Apparently 324 metres is not the correct height because, as every Frenchman knows, that includes the radio antenna on the top. Or something like that. The point is, they engaged in my presentation because they had to spend a moment or two thinking and then declaring what they thought by pressing a button on the clicker.

That’s what clickers can do for you.

And once you’ve got them engaged and participating, voila, they’re ready to learn the important stuff.