Tag: clickers

I’m working with a veteran gen-ed astronomy (#astro101) instructor to make his classroom more learner-centered. We’re working hard on effective clicker implementation. The benefit of using clickers for think-pair-share (TPS) questions is the instructor can use the students’ votes to guide the instruction.

clicker_iclicker_UIC — i>clicker receiver and clicker (sorry, can't find credits for this pic.)

If everyone gets a question right, just confirm the answer and move on – don’t waste valuable class time re-teaching something everyone already knows! Conversely, if the students have no clue what the answer is and simply guess, you’d expect 20% for each choice A-E, 25% each if there are 4 choices, and so on. If that’s how they vote, either there’s something wrong with the question (a critical typo, perhaps) or the students haven’t learned the concept yet. Teach it again BUT NOT JUST LOUDER. Teach it again in a different way.

The “sweet spot” is when there’s a nice split between 2 or choices. The students have thought hard enough to formulate and pick the choice they feel is correct, which means they’re prepared to interact with their peers. In cases like this, we ask them to “turn to your neighbours and convince them you’re right.” Then you sit back and let them teach themselves. Ahhh.

(Well, actually, you shouldn’t sit back. You should wander around the room and eavesdrop – you’re going to hear some great ideas you can use for choices on the final exam!)

The hard part for instructors is knowing when to move on or when to get the students to discuss the question. Is 90% correct enough? Yes, probably. What about 80%? What about 60%?

In today’s astronomy class, the instructor asked the students a TPS question and the distribution of votes was A 0, B 0, C 67%, D 20%, E 13%. The instructor wasn’t overjoyed, but 67%? That means 2/3 of the students got it, right?

Wrong. Some knew the answer. And the rest guest. Er, guessed.

I did a little thought experiment with the instructor afterwards. “Suppose only half the students knew the answer and the rest just guessed. What vote distribution would you get?”

“Er, 50% then 10% for each choice, so a 60 and 10’s.”

“Great,” I said. “Suppose 2 of the 5 choices were obviously wrong. Then what.”

He thought for about 2 seconds. “67-17-17.” Our numbers from that today. “Oh.”

That’s right, when there are only 3 valid choice and only half the students know the answer, you still get about 67% success. And you might be tempted to move on even though half the students don’t know what you’re talking about!

That got me thinking – suppose fraction f of the students know the correct answer and the rest guess. What do the clicker vote distributions look like? I cast a spell with Excel (I’ve finally reached novice Excel spellcaster) and found these results:

ClickerDistWhenfKnow_5choices — Distribution of votes when fraction f of students know the correct answer is A and the rest of the students make a random guess. Each set of 5 bars show the votes for A, B, C, D, E.

(Quick limit test that us math-types do: when no one knows and f=0.0, the votes are 20% for each choice. And when everyone knows, it’s 100-0-0-0-0. Got it.)

For example, when the peak vote is 60%, only 50% of the students actually know the answer. And it gets worse when there are fewer choices (or equivalently, when you can eliminate some of the 5 choices because they’re obviously wrong.) Here are the distributions when there are 4 choice and 3 choices:

ClickerDistWhenfKnow_4choices — Distribution of votes when fraction f of students know the correct answer is A and the rest of the students make a random guess. Each set of 4 bars show the votes for A, B, C, D.

ClickerDistWhen4Know_3choices — Distribution of votes when fraction f of students know the correct answer is A and the rest of the students make a random guess. Each set of 3 bars show the votes for A, B, C.

This last chart shows our 67-17-17 vote distribution corresponding to only 50% of the students knowing the right answer.

This isn’t ground-breaking research. I bet many clicker users have done this, too. Or at least, worked out a few special cases.

The moral of the story, though: the fraction of students who choose the correct answer is always higher than the fraction of students who know the correct answer. Don’t move on to the next topic unless you get a very strong peak.

What’s your threshold for moving on or doubling-back with a pair-share?

But did they learn anything?

December 13, 2010

Peter Newbury

The course transformations I work on through the Carl Wieman Science Education Initiative (CWSEI) in Physics and Astronomy at UBC are based on a 3-pillared approach:

figure out what students should learn (by writing learning goals)
teach those concepts with research-based instructional strategies
assess if they learned 1. via 2.

Now that we’ve reached the end of the term, I’m working on Step 3. I’m mimicking the assessment described by Prather, Rudolf, Brissenden and Schlingman, “A national study assessing the teaching and learning of introductory astronomy. Part I. The effect of interactive instruction,” Am. J. Phys. 77(4), 320-330 (2009) [link to PDF]. They looked for a relationship between the normalized learning gain on a particular assessment tool, the Light and Spectroscopy Concept Inventory [PDF], and the fraction of class time spent on interactive, learner-centered activities. They collected data from 52 classes at 31 institutions across the U.S.

The result is not a clear, more interaction = higher learning gain, as one might naively expect. It’s a bit more subtle:

Prather_AmJPhys_2009-F7_UBC — Learning gain on the LSCI and Interactive Assessment Score, essentially the fraction of class time spent on interactive instruction. Each point represents one class with at least 25 students. (Prather et al, 2009) Our UBC result from the Sep-Dec 2010 term is shown in green.

The key finding is this: In order to get learning gains above 0.30 (which means that over the course of the term, the students learn 30% of the material they didn’t know coming in) — and 0.30 is not a bad target — classes must be at least 0.25 or 25% interactive. In other words, if your class is less than 25% interactive, you are unlikely to get learning gains (yes, as measured by this particular tool) above 30%.

Notice it does not say that highly interactive classes guarantee learning — there are plenty of highly-interactive classes with low learning gain.

Back in September, I started recording how much time we spent on interactive instruction in our course, ASTR 311. Between think-pair-share clicker questions, Lecture-tutorial worksheets and other types of worksheets, we spent about 35% of total class time on interactive activities.

We ran the LSCI as a pre-test in early September, long before we’d talked about light and spectroscopy, and again as a post-test at the end of October, after the students had seen the material in class and in a 1-hour hand-on spectroscopy lab. The learning gain across 94 matched pairs of tests (that is, using the pre- and post-test scores only for students who wrote both tests) came out to 0.42. Together, these statistics put our class nicely in the upper end of the study. They certainly support the 0.30/25% result.

Cool.

Okay, so they learned something. How come?

The next step is to compare student performance before and after this term’s course transformation. We don’t have LSCI data from previous years, but we do have old exams. On this term’s final exam, we purposely re-used a number of questions from the pre-transformation exam. I just need to collect some data – which means re-marking last year’s final exam using this year’s marking scheme. Ugh. That’ s the subject of a future post…

Wasn’t expecting Him in class

December 2, 2010

Peter Newbury

astro 101 clickers 2 Comments

In the #astro101 class I’m working on, we just reached the “what is life” section. Great timing, considering the new @NASA astrobiology discovery of a bacteria that, unlike every other living creature, uses arsenic instead of phosphorus in its DNA.

We were going to have a PPT slide that listed 4 “generally agreed-upon” characteristics of life

DefiningLifePPT — Four “generally agreed-upon” characteristics of “life”. Kind of a boring PPT slide for such an intersting topic, no?

<Yawn> I suggested to the course instructor we switch it into a #clicker question, to get the students to critically think about each characteristic and then compare them to what they think “life” means:

DefiningLifeClicker — The same content posed as a clicker question to, er, lure the students into thinking about each characteristic.

I intentionally added the last choice “E) other ______” so students could add their own ideas. The instructor and I talked about it ahead of time, and agreed that if students chose E), we’d invite them to share their ideas with the class.

Fast forward to class. We pose the question, not as a think-pair-share sequence but just inviting them to discuss it with their neighbours. Then the students voted.

l1012011155_c5 — Students’ votes for A, B, C, D, E.

Excellent – 4 others. Wonder what they are?

“What other characteristics should a life form have?”

Then the shocker. From the back of the room comes

“God!”

In hindsight, we should have expected that! But we weren’t prepared for it. Kudos to the instructor, though: without even a pause, she replied, “Well, we’re not going to add religion and philosophy to this science class. Okay, let’s see how these 4 characteristic apply…”

The student’s answer was a great one. It told us he’d thought about the question we posed and compared it to his own knowledge, experience and beliefs. Who could ask for anything more? Be warned, though: if you want to invite your students to bring their religion into your astronomy class, be prepared – you can’t just wing it. (I did that once. Big mistake. Made me look pretty – no, make that very – ignorant.) And if you’re not familiar with the spectrum of religious beliefs in your classroom, you might want to reconsider the conversation before you start it. Why not be up front about it with your students:

Whenever people talk about the origin of life, some will undoubtedly want to include their religious beliefs. In this class, though, we’re going to stick to the scientific aspects of the discussion, the aspects that can be predicted, observed, proved or disproved by the scientific method. Now, about those scientific characteristics of life…

Navigation