A misconception about extrasolar planets

A couple of weeks ago in the introductory “Astro 101” class I work in, the instructor and I confirmed that many students hold a certain misconception. I was, still am, pretty excited about this little discovery in astronomy education. If my conversations over the following few days had turned out differently, I probably would be writing it for publication in the Astronomy Education Review. Maybe I still will. But for now, here’s my story.

Our search for life in the Universe and the flood of results from the Kepler Mission have made the discovery of extrasolar planets an exciting and relevant topic for introductory “Astro 101” courses and presentations to the general public.  Instructors, students, presenters and audiences latch onto “the transit method” of detection because it is so intuitive: when an extrasolar planet passes between us and its star, the planet temporarily blocks some star light and we detect a dip in the brightness of the star. The period and shape of the dips in the record of the star’s brightness encode the characteristics of the planet.

When an extrasolar planet passes between us and its star (when it "transits" the star) we detect a dip in the brightness of the star. (Kepler/NASA image)

Our students do a nice 50-minute, hands-on lab about how to decode these “light curves” which I hope to share at the ASP 2011 conference (#ASP2011 on Twitter) in July. In a class following this lab, the instructor posed the following think-pair-share clicker question. We wanted to assess if the students remembered that the size of the dip is proportional to the area of the star blocked by the planet’s disk, which scales as the square of the diameters:

Clicker question to assess the students' grasp of the transit method of detecting extrasolar planets.

The bars in this histogram record the number of students who chose (from left to right) A to E:

Students' responses for (left to right) choices A to E to extrasolar planets clicker question.

About 60% of the class chose answers (C and E) with a 1% drop in brightness, the correct drop, and about 40% chose answers B and D with a 10% drop. This second group didn’t remember the “proportional to area” property. So, not stunning results, certainly a good candidate for pairing and sharing.

The misconception

What is stunning, though, and the source of my excitement, is that 97% of the class feels you see a black spot moving across the star. Which is not true! We only detect the drop in the brightness of the star. We can’t even see the disk of the star, let alone a tiny black spot!

Okay, okay before you jump to the students’ defence, let me (with the help of my great CAPER Team colleagues) jump to the students’ defence:

  1. The question says, “…by observing it pass in front of the distant star.” Of course the students are going to say we see a dark spot – that’s what we just told them! Perhaps I should be worried about the 3% who didn’t read the question properly.
  2. The question is vague about what we mean by “size.” Diameter? Area? Volume? Mass? “The star’s diameter is 10 times bigger than the planet’s diameter” is a much better question stem.
  3. My colleague Aaron Price points out
  4. Astronomers may not see a “dot” crossing the star right now, but they can see something comparable. Through speckle imaging, radial topography and optical interferometry we have been able to see starspots for decades. CHARA’s recent direct observations of a disk of dust moving across epsilon Aurigae shows what is being done right now in interferometric direct imaging. I predict within 10 years we’ll have our first direct image of a “dot” in transit across another star.

  5. Aaron, Kendra Sibbernsen and I all agree that the word “see” in “What would you see?” is too vague. The question I wanted to ask should have used “observe” or “detect”. Kendra suggested we write “A) a dark spot visibly passing in front of the star” and perhaps following up the question with this one to poke explicitly at the potential misconception:

With current technology, can astronomers resolve the dark spot of an extrasolar planet on the disk of a star when it is in transit? (T/F)

Was there a misconception?

Did the students reveal a misconception about transiting extrasolar planets. Nope, not at all. It’s not like they took the information we gave them, mixed it with their own preconceived notions and produced an incorrect explanation. Instead, they answered with the information they’d been given.

A teachable moment

It seems that we’re not being careful enough in how we present the phenomenon of transiting extrasolar planets. But as it turns out, this is a teachable moment about creating models to help us visualize something (currently) beyond our reach. We observe variations in the brightness of the star. We then create a model in our mind’s eye — a large, bright disk for the star and a small, dark disk for the planet — that helps us explain the observations.

This is a very nice model, in fact, because it can be extended to explain other, more subtle aspects of transiting extrasolar planets, like a theoretical bump, not dip, in the brightness, when the planet is passing behind the star and we  see detect extra starlight reflected off the planet. The models also explains these beautiful Rossiter-McLaughlin wiggles in the star’s radial velocity (Doppler shift) curve as the extrasolar planet blocks first the side of the star spinning towards us and then the side spinning away from us.

These wiggles in the radial velocity curve are caused by the Rossiter-McLaughlin effect (from Winn, Johnson et al. 2006, ApJL)

Want to help?

If you’re teaching astronomy, you can help us by asking them this version, written by Kendra, and letting me know what happens.

An extrasolar planet passes in front of its star as seen from the Earth. The star’s diameter is 10 times bigger than the planet’s diameter. What do astronomers observe when this happens?

A)  a dark spot visibly passing across the disk of the star
B)  a 10% dip in the brightness of the star
C)  a 1% dip in the brightness of the star
D) A and B
E) A and C

In conclusion

I don’t think this qualifies as a misconception, not like the belief that the seasons are caused by changes in the distance between the Earth and the Sun. We’re just need to be more careful when we teach our students about extrasolar planets. And in more-carefully explaining the dips in the light curve, we have an opportunity to discuss the advantages and disadvantages of using models to visualize phenomena beyond our current abilities. That’s a win-win situation.

Thanks to my CAPER Team colleagues Aaron, Kendra and Donna Governor for the thoughtful conversations and the many #astro101 tweeps womanastronomer, erinleeryan, uoftastro, jossives, shanilv and more who were excited for me, and then patient with me, as I figured this out.

#eqjp, a teachable moment

In my current assignment through the Carl Wieman Science Education Initiative in Physics and Astronomy at UBC, I’m working closely with a senior astronomy professor to help him better teach his general-education “Astro 101” course. It’s a mixture of providing resources, mentoring, helping him clarify what he wants the students to learn, and coaxing (sometimes dragging – he’s a great sport!) his teaching to a learner-centered approach.

Today was supposed to be the first class in the last, big section of the course, comparative planetology. That is, the characteristics of the planets and other bodies in our Solar System and, more importantly, what their similarities and differences tell us about the formation of Solar System some 4.5 billion years ago. Traditionally, one follows the textbook’s lead. Chapter 10: Mercury. Chapter 11: Venus,… Chapter 15: Saturn,… Chapter 20: Other Crap, Chapter 21: [finally!] Formation. And by this time, nobody remembers Mercury, Venus, or gives a damn. I’m glad to say we long ago scrapped that approach and instead, focus on the gathering and analyzing the evidence that points to a single formation event. Our learning goal states that a student will be able to

deduce from patterns and properties of the planets, moons, asteroids and other bodies that the Solar System had a single formation event.

Where was I? Oh, right, teachable moment.

Last night (March 10), there was a massive earthquake in Japan. Magnitude 8.9, one of the biggest earthquakes recorded. The ensuing tsunami(s) devastated parts of Japan. I pay attention to these things, perhaps more than others, because my home, Vancouver, is on the list of places expecting The Big One. And we can be hit by tsunamis caused by earthquakes around the ring of fire. Thankfully, the west coast of Canada and the U.S. were spared this time.

It occurred to me, on the bus ride to work this morning, we could use last night’s earthquake in class today. Seismic activity tells us about the structure and evolution of the Earth. Similar signs of earthquakes and volcanoes on other planets, or lack thereof, tell us about their structure and evolution. Not seeing volcanoes on a planet is just as telling as seeing them. Using the earthquake to introduce this last arc in the course would set the tone for the next month of classes: we don’t care about the exact surface temperature on Mercury or the exact density of Neptune. We care about patterns in the physical properties of the planets. And we care about how we find, collate and reconcile those patterns.

Shortly after this “A-ha!” moment, my brain countered with, “Is this a teachable moment. Or are you exploiting the earthquake because you can’t think of an interesting way to teach comparative planetology?”

So I tweeted…

…and, as usual, was overwhelmed by the quick and intelligent response of the great tweeps who follow me. Thanks @TanyaCNoel, @penmachine, @snowandscience, @cpm5280, @derekbruff, @erinleeryan, @cosmos4u. The overwhelming advice was take advantage of the teachable moment:

Good idea. Understanding is always helpful.
teachable moment. everyone’s talking about it anyway…
Definitely a teachable moment

I’m also thankful to @ptruchon for putting words to something that bothered me:

Tough one…Do some of them have family in Japan? If so, are they ok?

So, I went for it. And by went for it, I mean I decided to convince the prof to use the earthquake in today’s class. I proposed he could run the “Earth’s Changing Surface” lecture-tutorial but he decided against it. Instead, he used the earthquake to segue from “here are the 3 or 4 key patterns that support a single formation event” to “how do we know all that, anyway?” Through open questions  like, “What does the earthquake tell us about the structure of the Earth?” and “What does this picture [of Mars’ Olympus Mons] tell you about this planet?” he lead a nice discussion with the 170-or-so students in class today. Many students, men and women, from the front and the back of the lecture hall, participated.

A very successful class, in my opinion, one that demonstrated to me and himself and the students, how “agile” this prof is getting. I was proud that we were able to adapt our presentation so quickly and help the students learn about something they care about.

P.S. A special hat-tip to @cpm5280 who reminded me about that this earthquake was predicted, yes predicted, by the Super Moon wingnuts. I gave the prof a quick summary, just in case. And sure enough, at the end of class, a gaggle of students came down and asked him if he knew anything about the Moon being super-close on March 19. He hit them with a few, key scientific facts (in particular, that because gravity follows an inverse-square law, the tiny decrease in distance won’t do very much) and told them that the whole earthquake-prediction thing was, “a load of crap.” He used their language and they, like, totally got it.

Evidence of Learning in Astronomy

Throughout the Sep-Dec, 2010 term, I worked with an astronomy instructor to create a more learner-centered classroom. As I described elsewhere, we spent just over one third of the instructional time on interactive activities: think-pair-share using clickers, lecture-tutorial worksheets, ranking tasks and a couple of predict-observe demonstrations. And it resulted in a learning gain of 0.42 on a particular assessment tool, the LSCI. That means the students learned 42% of the concepts they didn’t already know at the beginning of the term. That’s not bad — we’re pretty happy with it.

So, students can learn in a learner-centered classroom. But maybe they can learn in a more traditional classroom, too.

We don’t have LSCI data from previous years (note to self: think ahead! Collect standardized assessment data on classes before attempting any transformations!) To investigate if transforming the instruction class makes any difference, we re-used, word-for-word, a handful of questions from the same instructor’s 2008 Final Exam (pre-transformation) on this term’s Final Exam: 10 multiple-choice questions and 4 longer-answer questions. We made sure the questions assessed the concepts we covered in 2010 in sync with the learning goals.

I extracted students’ marks on these 14 questions from the 2010 exams (N=144). And from the old, 2008 exams (N=107), being sure to re-mark the longer-answer questions using the 2010 rubric. (Note to self #2: buy aspirin on the way home.)

What were we hoping for? Significant increase in student success in the transformed,  learner-centered course.

How I wish I could report that’s what we found. But I can’t. Because we didn’t. Here are the results:

Students scores on questions used on both the 2008 and 2010 Final Exams in the introductory astronomy course, ASTR 311. Error bars are standard error of the mean.

There is no significant difference in student success on the 10 multiple-choice questions. Their scores on the entire exams are also the same, though the exams are not identical, only about 1/4 of the 2008 exam is re-used in 2010. Nevertheless, these nearly identical Exam scores suggest the populations of students in 2008 and 2010 are about the same.  There are are differences in the 4 long-answer questions: the 2008 students did better than their 2010 counterparts.

Two things jumped out at me

  1. Why did they do so much better on the long-answer questions? I said we used the same marking rubric but we didn’t use the same markers. A team of teaching assistants marked the 2010 exams; I(re) marked the 2008 exams. The long-answer questions are work 10 marks, so a little more (or less) generosity in marking – half a mark here, half a mark there – could make a difference. I really need to get the same TAs to remark the 2008 exams. Yeah, like that’s gonna happen voluntarily. Hmm, unless there’s potential for a publication in the AER
  2. Why, oh why, didn’t they do better this year? Even if we omit the suspicious long-answer marks and look only at the multiple-choice questions, there is no difference. Did we fail?

No, it’s not a failure. The instructor reduced her lecturing time by 35%. We asked the students to spend 35% of their time teaching themselves. And it did no harm. The instructor enjoyed this class much than in 2008. We had consistent 75% attendance (it was much lower by the end of the term in 2008) and students were engaged in material each and every class. I think that’s a success.

The next step in this experiment is to look for retention. There is evidence in physics (see Pollock & Chasteen, “Longer term impacts…” here) that students who engage in material and generate their own knowledge retain the material longer. With that in mind, I hope to re-test these 2010 students with LSCI in about 3 months, after they’ve had a term to forget everything. Or maybe not…