Category: UBC Okanagan

Supporting SoTL

Scholarly teaching. Education research. Scholarship of Teaching and Learning. These are all activities related to applying valid research methods – typically developed in other disciplines – to study teaching and learning.

For faculty members who’s merit, tenure, and promotion is based, in part, on their research output, publishing articles about education can’t hurt but it may not be seen as important as their disciplinary research. UBC, like a growing number of universities, has a tenure-track stream of Assistant, Associate, and (full) Professor of Teaching. We call it the Educational Leadership stream because success and promotion requires demonstrating impact and leadership beyond your classroom. For faculty in this stream, engaging in SoTL is a powerful way to demonstrate that leadership.

It’s my Centre for Teaching and Learning’s mission to “promote, inspire, and support excellence, leadership, scholarship, and technologies in teaching and learning.” I find supporting scholarship is one of most difficult part of our mission because when we start talking about research, each faculty member immediately snaps to the kinds of disciplinary research they do – if any – and tries to force education into that methodology. I struggle to support them because (i) I don’t know what kind of research they do and (ii) I’m most familiar with research methods found in STEM.

I’m writing this post because something happened last week, something good, that’s changed my approach and, I hope, the success of the faculty members I work with. Here’s the story. Dr. Jasmin Hristov, a research-stream Assistant Professor in the Department of History & Sociology, Irving K. Barber School of Arts and Sciences gave me her permission to tell it.

Professor Hristov teaches upper-level sociology. She plans to bring in a series of guest speakers via video conference and asked if she could use my Centre’s workshop room. “Yes, of course,” I replied. And then, thinking about my Centre’s mission, I added, “You’re doing something innovative – would you be interested in talking about how you could study whether or not it’s effective?” She was, and we met.

First, Professor Hristov described her motivation: introduce the students to six experts from around the World, with careful attention to diversity of gender, race, location, and rank. For each guest speaker, the students do some background reading, prepare questions to ask the speaker, and lead a discussion. After class, the students write a reflection about the experience.

“How can we tell if it was effective? How can we tell if students learned anything?”

We nearly got lost down a dead end. Professor Hristov: “I’ve taught this course before without the video conferencing but with different students and, obviously, without the reflection.” Both of us nearly concluded, “Without a control group to compare grades against, I don’t see how we can study this.”

We didn’t go there, though, because serendipitously, I started the conversation with,

How can we find evidence of impact?

This question opened up whole new ways of thinking, without sending us on that narrow “research = A/B study with statistical significance” path. It led quickly to a couple of possibilities that could produce interesting results that don’t rely on the success or failure of p < 0.05.

Text analysis of students’ reflection

4-page reflection × 6 reflections × 30 students = huge amount of text

Imagine examining all that text with powerful tools like Voyant or NVivo. Will students naturally comment on the diversity of the speakers? That was one of the elements deliberately built into this intervention, recall. Do they need a prompt? Not a heavy prompt like, “Please comment on the diversity of the speakers.” That will only get the answers the students think Professor Hristov wants to hear. Something more subtle, like, um, not sure yet.

But imagine the kind of evidence of impact she could include in the SoTL article:

“I carefully chose the speakers to expose my students to a wide range of races, locations, genders, and ranks. In their reflections, students made the following associations…”

This isn’t cherry-picking an individual student’s comments – that’s a helpful exemplar or supporting anecdote but it’s not evidence. Instead, we have legitimate connections and insight students are making.

Quantitative analysis of reflection grades

Just because we can’t do a controlled A/B study doesn’t mean we can’t do quantitative analysis. Imagine this: imagine we compare the students’ marks on the reflections with their marks on the rest of the course. The reflections are worth around 1/3 of the total mark, so the reflections are worth enough that students will put legitimate care and effort into them. In other words, the reflections are not some incidental marks students can blow off, and they’re not so important that nothing else matters in the course. I made up some data (thx, RANDOM.org) to see what kinds of conclusions we could make (click to enlarge):

Hypothetical student marks on the reflection and other course assignments, with a range of correlations and conclusions about impact. (Data via RANDOM.org. Graphic: Peter Newbury)

The left graph shows there’s a relationship between the students’ success on the reflections and the rest of the course. Do the reflections help them succeed with the other assignments? Do the other assignments help them write better reflections? Can’t tell. Better look at the text analysis…

The center graph isn’t telling a compelling story. Success on the reflections doesn’t seem to have any connection to success on the rest of the course. We can probably conclude the same about what the students are getting out of the video conferences. Time to rethink how the video conferences are integrated and supported.

The right graph is a worst-case scenario: success on the reflections comes at the expense of the their success in the rest of the course. Oh c’mon, this would never happen, right? Well, I’ve seen courses where there’s a “capstone project” that takes all the students’ time. If the capstone is that important, it should probably represent a significant fraction of the overall course mark, so success on the capstone guarantees success in the course. I’ve also seen cases where success on the capstone requires sacrificing the other courses you’re taking – time for the Department Head to get the course instructors together to coordinate their assignments!

No matter the scenario, there’s something here for Professor Hristov to share in the discussion of her SoTL paper. The conclusions will be useful to others thinking about integrating video conferencing into their courses.

Evidence of Impact

This will be my new conversation starter when promoting, inspiring, and supporting scholarship. It’s also a good prompt for the faculty members, themselves, who want to (need to?) demonstrate educational leadership. This prompt invites us to be curious and creative, instead of trying to jam teaching and learning into the same research methods that we’re familiar with from disciplinary research.

Open Classroom pop-up thank-you card

Every Fall, I follow along as Derek Bruff @derekbruff tweets out inspiring stories from the open classroom event his Center for Teaching runs at Vanderbilt. Course instructors from across campus volunteer to open their classrooms and welcome their peers to come observe. While we may have 25, 60, 300, or more students in our classrooms, it’s rare to have a colleague, and open classroom events provide an opportunity for all the educators in the room – the ones at the front and the ones at the back – to share some formative feedback.

We hosted our first Open Classroom Week at UBC Okanagan October 1-5, 2018. Twelve course instructors from across the campus, across disciplines, and from 1st-year to 4th-year invited their peers into their classrooms.

Opening your classroom to your colleagues takes courage and confidence and demonstrates educational leadership. So, I wanted to thank those twelve course instructors. Sure, I could send SW-S, RT, RP, WSM, CL, RF, GD, TE, TF, NL, CS, and AK a letter (or a letter to their Department Heads) on Centre letterhead, formally thanking them for participating in the event. But I wanted something they could put on the shelf in their office so remind them, and any visitors, that they did something valuable. Combine that with my obsessi–, er, interest in pop-up cards and you get this:

The pop-up thank-you card I made for the UBC Okanagan course instructors who opened their classrooms and welcomed their peers and colleagues to come and observe. A template and instructions available below so you can make your own.
The pop-up thank-you card I made for the UBC Okanagan course instructors who opened their classrooms and welcomed their peers and colleagues to come and observe. A template and instructions available below so you can make your own. (Photo: Peter Newbury CC-BY)

Do it yourself

Want to make one for your Center? Here’s the PPT file I used to create the card, plus a set of directions for editing the text, printing the card, and making it. I think the instructions are clear but by the time I wrote them, I’d already made 5 prototypes and then the dozen cards I gave to my UBC Okanagan colleagues so I could pretty well do it in my sleep. If you get stuck, feel free to tweet me at @polarisdotca. And then send me a picture of your finished card (and permission to share it)!

Anatomy of a 400-seat Active Learning Classroom

(This is adapted from a poster I presented at the 2018 Society for Teaching and Learning in Higher Education (STLHE) Conference, Université de Sherbrooke, June 20-22, 2018.)

(Photo courtesy of Ashlyne O’Neil. Thanks @ashlyneivy!)

Designing a Large, Active Classroom

As class size increases, instructors face an increasingly difficult challenge. There is clear evidence that more students are more successful in classes with active learning.[1] Yet the work required to facilitate active learning – logistics, providing feedback, supporting and interacting with individual students – increases with class size. And despite the importance of the design of learning spaces,[2] large classrooms often impede student-student and student-instructor interactions.

At UBC’s Okanagan campus, I was invited to advise the architects and campus planners on the design a new 400-seat classroom.

Design Principle:
Eliminate everything that hinders
student-student collaboration and
student-instructor interaction.

My poster uses a giant 6-page “book” (you can see it drooping slightly in the center of the poster in the picture above) to highlight different features and characteristics of the design:

Student flow: Main entrances to the classroom are at the middle of the room. Students flow in and downhill toward the front. Sitting at the back takes deliberate effort. Students can discretely enter and exit without disrupting the class or the instructor.
Main entrances to the classroom are at the middle of the room. Students flow in and downhill toward the front. Sitting at the back takes deliberate effort. Students can discretely enter and exit without disrupting the class or the instructor.
Accessible seating: Fully 20% of seating – roughly 90 locations – are accessible to students using wheelchairs. They can sit in groups with their peers at prime locations, instead of being isolated or confined to designated seats.
Fully 20% of seating – roughly 90 locations – are accessible to students using wheelchairs. They can sit in groups with their peers at prime locations, instead of being isolated or confined to designated seats.
Network of aisles: A network of aisles throughout the classroom allows instructors and teaching assistants to get face-to-face or within arm’s reach of every student. Wireless presentation system allows instructors to teach from any location and project any student’s device.
A network of aisles throughout the classroom allows instructors and teaching assistants to get face-to-face or within arm’s reach of every student. Wireless presentation system allows instructors to teach from any location and project any student’s device.
Group work with whiteboards: Students on narrower front desks swivel around to work with their peers on wider desks. With 150 whiteboards scattered throughout the room, groups can be collaborating within seconds of their instructor saying, “Grab a whiteboard and…”
Students on narrower front desks swivel around to work with their peers on wider desks. With 150 whiteboards scattered throughout the room, groups can be collaborating within seconds of their instructor saying, Grab a whiteboard and…
Lighting: Separate front, middle, back lights create smaller classrooms for 250 and 100 students.
Separate front, middle, back lights create smaller classrooms for 250 and 100 students.
Prep room: Prep room is accessible from outside the classroom so instructors can prepare before and after class. Includes sink, glassware drying rack, storage cabinets, lockable flammable solvent cabinet, fume hood, chemical resistant countertops, first aid kit, demo cart.
Prep room is accessible from outside the classroom so instructors can prepare before and after class. Includes sink, glassware drying rack, storage cabinets, lockable flammable solvent cabinet, fume hood, chemical resistant countertops, first aid kit, demo cart.

Design Features Promote Collaboration and Interaction

Design Features Promote Collaboration and Interaction

  • The classroom is gently tiered so students farther back can see the front. There are 2 desks on each tier. The front desk is wide enough to hold a notebook and laptop. The rear desk is nearly twice as wide, allowing the front student to swivel around and work with their peers in the rear desk.
  • Swivel chairs on wheels allow students to easily move and work with others around them.
  • The front desk on each tier has a modesty screen. There are deliberately NOT modesty screens on the rear desks, allowing students on the front desk to swivel around to the rear desk without smashing their knees or having to sit awkwardly.
  • There are power outlets for every student under the desktop, leaving the work surface unbroken and smooth for notebooks, laptops, and whiteboards.
  • When the instructor or teaching assistant stands in the aisle in front of the front desk, they can speak face-to-face with the 1st row of students, and are within arm’s reach of the 2nd row. From the aisle on the back of this set of four rows of desks, the instructor or teaching assistant is face-to-face with students in the 4th row and within arm’s reach of the 3rd row.

Optimizing Visibility of the Screen

A slightly curved screen at the front of the classroom is large enough to display two standard inputs. A third projector can display a single image across the screen. The screen is about 7 or 8 feet above the floor, so the instructor at the front does not cast a shadow on the screen or look directly into the projectors (housed in a 2nd floor projection room at the back of the classroom.) The size and curvature of the screen ensure all but the very front-left and front-right seats have views of the screen within UBC’s guidelines.

Does the Design Enhance Learning?

We are studying the impact of the design by comparing data collected before and after course instructors teach their courses in the 400-seat classroom, including

  • distributions of final grades and grades on in-class activities like peer instruction (“clicker”) questions and group work sheet
  • drop, fail, withdrawal (DFW) rates
  • locations of the course instructor and teaching assistants at 2-minute intervals throughout the class period
  • what the instructor is doing (lecturing, writing, posing questions,…)  and what the students are doing (listening, discussing peer instruction questions, asking questions,…) using  the Classroom Observation Protocol for Undergraduate STEM (COPUS)3,4

COPUS captures what the instructor and what the students are doing during the class. There is a clear difference here between a traditional, lecture-based course and a course that uses active learning. (Graphic by CWSEI CC BY NC)

Acknowledgements

My thanks to Dora Anderson, Heather Berringer, Deborah Buszard, Rob Einarson, W. Stephen McNeil, Carol Phillips, Jodi Scott, and Todd Zimmerman for the opportunity to help design to this learning space.

Blueprint and visualizations by Moriyama & Teshima Architects. Used with permission.

References

1 Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410-8415. doi.org/10.1073/pnas.1319030111
2 Beichner, R., Saul, J., Abbott, D., Morse, J., Deardorff, D., Allain, R., … & Risley, J. (2007). The Student-Centered Activities for Large Enrollment Undergraduate Programs (SCALE-UP) project, a peer reviewed chapter of Research-Based Reform of University Physics. College Park, MD: Am Assoc of Physics Teachers.
3 Stains, M., Harshman, J., Barker, M. K., Chasteen, S. V., Cole, R., DeChenne-Peters, S. E., … & Levis-Fitzgerald, M. (2018). Anatomy of STEM teaching in North American universities. Science, 359(6383), 1468-1470. doi.org/10.1126/science.aap8892
4 Smith, M. K., Jones, F. H., Gilbert, S. L., & Wieman, C. E. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): a new instrument to characterize university STEM classroom practices. CBE-Life Sciences Education, 12(4), 618-627. doi.org/10.1187/cbe.13-08-0154

Navigation