Author: Peter Newbury

Over the last 9 months, I’ve had the privilege of working with a group of dedicated educators in the Dalhousie Faculty of Engineering to explore building a culture of integrity. In Part 1, I describe our faculty learning community where we read James Lang’s Cheating Lessons together. Part 2 is about a survey we ran with Engineering students and how we analyzed their responses. Here in Part 3, it all comes together with a resource we created for course instructors. Our analysis of responses and the resource for course instructors are still drafts. The co-authors are not ready to have their names appear in public so while I can’t give their names, I want to acknowledge these were collaborative projects and I wouldn’t have anything to write about without these colleagues.

Structures crack when they’re under too much stress. Redesign the structure to remove the excess stress and you stop it from cracking. The rise in academic integrity violations during the year of online teaching and learning is a crack in our teaching and learning practices.

Research into academic integrity¹ shows that the way to reduce academic dishonesty is to create a learning environment where students are successful without resorting to dishonest practices and even if they stray, there is no benefit. Creating and maintaining this environment requires ongoing discussion, collaboration, and cooperation of faculty, students, and staff. Together, we can build a culture of integrity that enables more students to be more successful, reduces exam stress and anxiety on both students and faculty, and puts our students on the path to becoming professional engineers.

In June, 2021, Co-author 1 and Peter Newbury surveyed and interviewed 19 Engineering students about their experiences with academic integrity. Co-author 2, Co-author 3, and Peter Newbury coded and categorized their 109 responses to three questions. The most frequent responses are shown here, together with strategies you can use in your courses. The full report and analysis of the students’ responses is available upon request.

1. Lang, J. M. (2013). Cheating lessons. Harvard University Press.

What are the ways I can make my course relevant to students?
You don’t need to link every concept and every example to a relevant application every single time. In fact, artificially forcing relevance into your lessons can demotivate students. Periodically (perhaps once each week) motivate a new concept or example with an application or scenario from engineering practice or a specific field of research. Wrap new concepts and skills in contexts that matter to students in this time, in this place, or in their personal lives so they immediately see the role and impact of engineering on their lives, family, friends, and community. Connect learning outcomes, concepts, skills, and examples to the Graduate Attributes. Emphasize to your students that demonstrating the learning outcomes means they are becoming engineers. Chat with the course instructors who teach courses after yours in the program so you can motivate new topics: “This will be important next Term when you study…” Chat with the course instructors teaching the other courses your students are currently taking. Identify examples, applications, scenarios, or cases that combine concepts and skills from several courses.

What are the ways I can make my course relevant to students?

You don’t need to link every concept and every example to a relevant application every single time. In fact, artificially forcing relevance into your lessons can demotivate students.

Periodically (perhaps once each week) motivate a new concept or example with an application or scenario from engineering practice or a specific field of research.
Wrap new concepts and skills in contexts that matter to students in this time, in this place, or in their personal lives so they immediately see the role and impact of engineering on their lives, family, friends, and community.
Connect learning outcomes, concepts, skills, and examples to the Graduate Attributes. Emphasize to your students that demonstrating the learning outcomes means they are becoming engineers.
Chat with the course instructors who teach courses after yours in the program so you can motivate new topics: “This will be important next Term when you study…”
Chat with the course instructors teaching the other courses your students are currently taking. Identify examples, applications, scenarios, or cases that combine concepts and skills from several courses.

What are the ways I can help my students handle the workload and manage their time?
Learning how to manage their time is a skill students are still developing. They may need guidance, especially in 1st year when experiences inside and outside the classroom are new to many. Don’t overload your students: design your course so students can be successful with 6 – 9 hours of work per week (including lectures), 9 – 12 hours per week for courses with labs. During the year of learning online, students appreciated it when course instructors suggested how to spend their time (“On Mondays, spend 2 hours watching videos and taking notes. On Tuesdays, begin the homework. On Wednesdays,…”) Continue to provide this guidance, even for in-person classes. Work with your Department and your students’ other course instructors to coordinate the timing of your assignments, projects, quizzes, and midterm exams. Try to ensure students have no more than one assignment due, project deadline, quiz, or midterm on any day across all their courses.

What are the ways I can help my students handle the workload and manage their time?

Learning how to manage their time is a skill students are still developing. They may need guidance, especially in 1st year when experiences inside and outside the classroom are new to many.

Don’t overload your students: design your course so students can be successful with 6 – 9 hours of work per week (including lectures), 9 – 12 hours per week for courses with labs.
During the year of learning online, students appreciated it when course instructors suggested how to spend their time (“On Mondays, spend 2 hours watching videos and taking notes. On Tuesdays, begin the homework. On Wednesdays,…”) Continue to provide this guidance, even for in-person classes.
Work with your Department and your students’ other course instructors to coordinate the timing of your assignments, projects, quizzes, and midterm exams. Try to ensure students have no more than one assignment due, project deadline, quiz, or midterm on any day across all their courses.

What are the ways I can communicate effectively and connect with my students?
The year of online learning demonstrated Brightspace can be an effective tools for communicating with students. Continue to use Brightspace to make important announcements (including the ones you made aloud during your in-person classes.) Students appreciate fewer, more comprehensive Brightspace announcements. Consider one (two at most) weekly announcement that includes all the information students need for the upcoming week. Invite people with first-hand experience (students for student events, practicing engineers for community events) to announce extra-curricular events and opportunities. Treat your students as professionals: they are on a path to becoming engineers and that path begins in 1st year.

What are the ways I can communicate effectively and connect with my students?

The year of online learning demonstrated Brightspace can be an effective tools for communicating with students. Continue to use Brightspace to make important announcements (including the ones you made aloud during your in-person classes.)

Students appreciate fewer, more comprehensive Brightspace announcements. Consider one (two at most) weekly announcement that includes all the information students need for the upcoming week.
Invite people with first-hand experience (students for student events, practicing engineers for community events) to announce extra-curricular events and opportunities.
Treat your students as professionals: they are on a path to becoming engineers and that path begins in 1st year.

Speaking of effective communication…

I recognize that course instructors are extremely busy, especially in the last few weeks before the first day of classes. To engage the most number of faculty in the limited amount of time, teaching resources need to be concise. During my time with the Carl Wieman Science Education Initiative, we aimed to create “2-pagers” – that’s long enough to say something useful but short enough faculty will read it.

I enjoyed the challenge of distilling 9 months of work into 2 pages and using all my Excel and PowerPoint skills. I’m quite happy with this draft of the resource, Building a Culture of Integrity in Engineering – What are the ways (pdf). My thanks, again, to my co-authors.

Building a Culture of Integrity, Part 2: What we learned from students

August 27, 2021

Peter Newbury

Over the last 9 months, I’ve had the privilege of working with a group of dedicated educators in the Dalhousie Faculty of Engineering to explore building a culture of integrity. In Part 1, I describe our faculty learning community where we read James Lang’s Cheating Lessons together. Part 2 is about a survey we ran with Engineering students and how we analyzed their responses. Part 3 brings it all together with a resource we created for course instructors. Our analysis of responses and the resource for course instructors are still drafts. The co-authors are not ready to have their names appear in public so while I can’t give their names, I want to acknowledge these were collaborative projects and I wouldn’t have anything to write about without these colleagues.

Guided by recommendations in James Lang’s Cheating Lessons, we followed up our conversations about academic integrity with faculty by continuing the conversation into the community, namely, the Engineering students.

The Associate Dean and I invited students to join an online focus group about academic integrity in Engineering at Dalhousie. We did not want these discussions to be about cheating but rather, the culture and environment. The discussion (and the survey for students who were unable to join the meeting) revolved around these three questions:

What would make the homework, lab reports, quizzes, exams, etc. meaningful to you, so that
you’d want to use your time and attention to do it yourself?
What are some factors making it difficult for students to do their own work?
When we have a strategy for building this culture, how do we connect with students so they
want to be a part of it?

We spoke with 7 students and 15 completed the survey. Between the notes I took during the meeting and the survey, we ended up with 109 responses.

Categorizing and analyzing the responses

Ever since my colleagues Beth Simon and Jared Taylor wrote their terrific article, “What is the Value of Course-Specific Learning Goals?” in 2009, I’ve been waiting for a project where I get to analyze student comments. It was important to me (and the Associate Dean) that we learn what the students are actually telling us, and not just seek confirmation of what we wanted to hear.

I recruited two members of the Cheating Lessons book club to help me sort and analyze the responses. My colleagues aren’t ready to have their names in public – this work and report is still a draft – so I’ll refer to them as F and P. Here’s what we did:

I created an Excel spreadsheet with three worksheets, one for each question. Each worksheet had one student response per row, with a unique identifier (so we could later talk about Response #16, etc.). I sent the spreadsheet to F and P.
Each of us independently read through the responses and created 3 or 4 “buckets” or categories of responses for each question. The idea is to let the categories emerge from the responses, rather than creating categories ahead of time (and risking simply confirming what we wanted to hear.)
We met to compare categories. Some were identical, some were different words for the same idea, and a few were different. We reached consensus on three categories for each question.
I reformatted the Excel spreadsheet, still with one student response per row and now 3 new columns, one for each category. I sent the spreadsheet to F and P.
Each of us independently re-read all the responses, assigning each response to one of the categories. F and P sent me their spreadsheets.
I merged all three spreadsheets together to show how each of us categorized each student response. I used Fleiss’ kappa to calculate the inter-rater reliability. It’s an index between 0 (no agreement) and 1 (total agreement) that normalizes for random agreement (3 monkeys would agree occasionally). That Wikipedia page, btw, is very good and I followed the step-by-step example.
We met to compare how we categorized the responses. We completely agreed on the majority, were split on some, and completely disagreed on a handful of responses. We discussed that handful of responses and how we interpreted the what the student said. A few “Ohh!” and “Ahh!” later, we’d reached consensus. We also agreed to categorize any split response in the majority category.

With every response now categorized, I repeated the inter-reliability calculation:

Question	Number of responses	Fleiss’ kappa	Interpretation
What would make the homework, lab reports, quizzes, exams, etc. meaningful to you, so that you’d want to use your time and attention to do it yourself?	48	0.745	Substantial agreement
What are some factors making it difficult for students to do their own work?	37	0.725	Substantial agreement
When we have a strategy for building this culture, how do we connect with students so they want to be a part of it?	24	0.483	Moderate agreement

I’m glad we had good agreement on question 1 (“meaningful”) and question 2 (“factors”). I’m not too worried about our lower agreement on question 3 (“how to connect”) as that was asking for their suggestions rather than learning about their experiences.

What did the students tell us?

When investigating inter-rater reliability and analysis of responses, I learned it’s good to present a “code book” along with the data that names the categories, defines the categories, and gives sample responses.

What would make the homework, lab reports, quizzes, exams, etc. meaningful to you, so that you’d want to use your time and attention to do it yourself?

We asked students, “What would make the homework, lab reports, quizzes, exams, etc. meaningful to you, so that you’d want to use your time and attention to do it yourself?” Twenty of the 48 comments are about relevance, 14 are about assessment and marks, and 14 are about support.

Category	Description	Sample Responses
Relevance	Make the work relevant to: (i) the material in the course, (ii) the material in other courses, (iii) industry, and (iv) career aspirations.	• I’m more inclined to do work when I’m also taught how it applies to industry or how it applies to other skills I will apply in the future as an engineer. • I think they typically mean a lot to me anyway, but to make it more meaningful, I feel like I would be more motivated to do them if it were related to solving real world problems that have had a big impact in the communities I’m a part of (for example, working on engineering projects that deal with the SDGs and so on).
Assessment, Marks	Ensure the reward (i.e., marks) justifies the effort expended in completing the assignment (i.e., the assessment).	• Sometimes work doesn’t feel like it’s worth the time (long assignment, takes all weekend for 1%, when another assignment is worth 10%). • Work is meaningful to me if there is a grade attached to it. I have had experiences where homework and labs are not for a grade which discourages students from doing the work since there are other classes that we have.
Support	Provide adequate preparation and support mechanisms so students feel capable and confident in undertaking the assignment on their own. This could involve ensuring the learning objectives of the assignment are thoroughly explained and understood.	• Let 1st years know they have support. • Having sufficient resources is important – e.g. when class is all theory but then homework is application and students don’t know how to do it, they look elsewhere (like Chegg) for help (e.g. a course with classes involving theory and simple examples, but homework that involves complex problems). Have assignments with a range of questions that build.

What are some factors making it difficult for students to do their own work?

Category	Description	Sample Responses
Workload, Time Management	Workload demands in a given course and throughout the program leading to stress on students to perform well and meet all course expectations. Student and professor time management issues.	• Something to note: it takes time to organize your time. Knowing assignment weights for prioritization, knowing deadlines, being able to find assignments easily, being able to find notes easily to solve problems, and even knowing what is expected on exams and how they will be formatted. • I think for all engineering students, the workload is very heavy which can make it difficult to complete all work individually. It is still possible to do it individually, but I believe that the workload does play a role in making it difficult for students to complete work individually.
(Lack of) Support	Issues with required support systems and course resources.	• I also think that an overall lack of support from professors and TAs makes it difficult for students to do their own work because they aren’t getting enough help with difficult subjects. The student to TA ratio is often too large for students to be able to comfortably ask meaningful questions. As a result, students are less likely to do their own work because they are confused about what is going on. • Although professors and faculty are making a valiant effort to provide extra support for students, I think some students still feel a bit overwhelmed and unsure how to get appropriate support online. Overall, online learning seems to be quite a bit more difficult for most students, especially the aspect of not being able to learn in a classroom or be surrounded by peers and professors, as that is the environment a lot of students thrive in most.
(Lack of) Motivation	Motivational issues for students.	• When instructors aren’t engaged, don’t appear to want to be there, then students are less engaged, more likely to get help from elsewhere. • When instructors are clear about expectations, students engage more, try to do it themselves.

When we have a strategy for building this culture, how do we connect with students so they want to be a part of it?

Category	Description	Sample Responses
Effective Communication (the Mode)	Engage in effective, well-thought-out communications with individuals and groups.	• Communicate through student leaders – students listen to them. Through Dal Eng Society. I think the strategy should be spread via the student representatives. • Orientation aimed mostly at 1st year, though programs have orientation for 3rd year students coming from Associated Universities. • Brief updates usually capture students’ attention better than long ones. So, if a strategy is made, whatever mediums it is transmitted over, I guess keep it concise and remind them of its purpose and incentive.
Ethics, Professionalism (the Message)	Appeal to the integrity of all involved in the engineering education process; draw on the P.Eng. Code of Ethics.	• Throughout the program we get many lectures about integrity (CPST, history of engineering, law and ethics). People understand that if an engineer makes a mistake it can cost lives. We learn about the different historical events and because there is a process in getting the P.Eng. stamp, I believe this process will give you engineers with integrity. I don’t believe that there is much more on the university side that can be done. At some point it is up to the person to decide what type of engineer/person they want to be. • Phrase it as something that will strengthen their career and not just create obstacles in school.
Collaboration, Community	Look for opportunities to encourage collaboration where permitted. Provide opportunities for students to create a sense of community within and outside the classroom.	• Make communal spaces where students can work together on assignments and projects, possibly grouped by discipline. • Round table discussions – especially in person. • Also hosting fun events so they can socialize with each other and become a greater community. Online classes have made students lose the sense of community within their faculty.

I’m really happy with how we carried out the analysis and also with the results. So are my co-authors, F and B. And so is the Associate Dean who immediately took the next step and asked me, “Okay, so what are the ways I can make my course more relevant to students?” That was the spark for the next step of this project: a campaign based on education. In Part 3 of this series, I’ll share the resource we created for course instructors in Engineering.

Building a Culture of Integrity, Part 1: “Cheating Lessons” book club

August 26, 2021

Peter Newbury

Dalhousie professional development teaching No Comments

Over the last 9 months, I’ve had the privilege of working with a group of dedicated educators in the Dalhousie Faculty of Engineering to explore building a culture of integrity. In this, the first of three posts about our work and its outcomes, I’ll describe our faculty learning community. Part 2 is about a survey we ran with Engineering students and how we analyzed their responses. Part 3 brings it all together with a resource we created for course instructors. Our analysis of responses and the resource for course instructors are still drafts. The co-authors are not ready to have their names appear in public so while I can’t give their names, I want to acknowledge these were collaborative projects and I wouldn’t have anything to write about without these colleagues.

Like every other Department, Faculty, School, and University, course instructors in the Faculty of Engineering at Dalhousie struggled with a rise in academic dishonesty during the Fall 2020 Term, the first full term forced online because of the COVID-19 pandemic. Rather than looking for surveillance solutions, the Associate Dean (“the AD”) was keen to play the long game and find ways to change the culture of the Faculty so that cheating wasn’t an option students considered, and if they did stray, it didn’t help them.

Jim Lang’s Cheating Lessons: Learning from Academic Dishonesty (2013) is the blueprint for what came next.

I pitched the idea to the AD that we should run a “book club” where a group of well-situated members of the Faculty read the book together. The AD liked it and immediately contacted each Department Chair inviting them to join the cohort and to bring 1 or 2 influential course instructors from their Department. By January, 2021, we had a cohort of about 15 people, including me, the AD, a good number of the Department Chairs, and a group of course instructors, every one of us dedicated to teaching and learning. With the support of the Dean, everyone received a copy of Cheating Lessons.

From January – April, 2021, I led four 60-minute, online conversations based directly on three critical questions Lang poses in Cheating Lessons.

Conversation 1: Why should students bother to memorize or learn or connect or do their own work when technology can often provide them with the information they need (more) quickly and efficiently?

Themes that emerged from reading Chapter 9 “On Original Work” and our conversation:

There is certain knowledge — vocabulary, mathematical skills, scientific knowledge — engineers need to recall quickly to begin analyzing, applying, and designing and afterwards, evaluating and checking solutions
The act of memorizing and recalling information helps students learn.
You don’t find novel problems and designs on the internet.

Conversation 2: Why should students bother to do their own work when others can do it better and more easily?

Themes that emerged from reading Chapter 4 “Fostering Intrinsic Motivation” and our conversation:

Intrinsic motivation increases as students more get hands-on experience with authentic engineering problems.
There is ample opportunity in the activities of the courses and program to create opportunities where students are intrinsically motivated to engage and learn.
Assessments, from homework problems to term projects, can be intrinsically motivating when they are grounded in the students’ knowledge, skills, and experiences.

Conversation 3: Why should students bother to complete an assignment on their own when three of them working together may complete it more effectively?

Themes that emerged from reading Chapter 7 “Instilling Self-Efficacy” and our conversation:

We need to convince students to invest the time and energy to learn, not to earn marks.
We need to find ways to boost their self-confidence. This might mean rewarding their progress and incremental successes, without punishing them for getting something wrong. It might be their perception (and experience) that getting the correct answer the only way to be successful.

Conversation 4: What do we do next?

Chapter 8 “Cheating on Campus” is very nearly a step-by-step guide for creating and communicating a program for academically honest education:

Begin the Conversation among the Faculty
Continue It Into the Community
Time It Well
Focus Academic Integrity Campaigns on Education, Not Ethics

I’m happy to say we followed this advice quite well. In Part 2, you can read about the academic integrity survey and focus groups we ran with Engineering students, and how we analyzed their responses. And in Part 3, I’ll share the resource we drafted with strategies for course instructors to promote integrity in the way they teach. The goal is to share this resource with every instructor in Engineering in August while their preparing their Fall courses.

Building a Culture of Integrity, Part 3: What are the ways?