Designing Assessments with A.I. in Mind


Though it is tempting to simply ban A.I. use in your courses in order to avoid making changes to your assessments and assessment structure, this approach misses out on opportunities to prepare and empower your students, as well as to create more effective assessments. Though you may, after careful consideration, ban its use (there are many good reasons to do so, pedagogical and otherwise), the fact remains that generative A.I. may quickly permeate everyday life and most professional work; all of us—students and instructors—must spend the time carefully weighing the pros and cons. If our goal is mitigation and deterrence, we want to deter students responsibly; that is, ensure they have some awareness of the tool such that they can understand its uses and limitations. If our goal is to allow the students to use the tools as course aids, we want to encourage responsibly; that is, prepare and guide students in the use of an ambiguous, understudied and under vetted tool for learning. A balanced interplay between deterrence and promotion are the two sides of the design coin when it comes to assessments and A.I., long-term.

The four broad strategies outlined below will help you chart a path forward. These strategies are as much about taking up pedagogical strategies to prepare and empower students generally, as they are about design thinking. Each broader strategy amplifies the effectiveness of the others. If you can identify and implement an idea or two from each of the broad strategies that seem doable to you, you will have put yourself and your students in a strong position to continue to meet these challenges over time—and they will keep on coming. (For example, the new version of ChatGPT, “GPT-4”, is now available for $20/month and includes features such as prompt generation and suggested replies. This cost and feature differential can further exacerbate inequities between students.)

Employ relational and transparent pedagogies

When education feels like a transaction, students will respond in kind by making transactionally motivated decisions, like trying to use A.I. to reduce time spent on required assessments, especially if they cannot understand the rationale behind the assessment in the first place. In whatever ways suit your pedagogical style, employ strategies that increase relationship-building and pedagogical transparency whenever you see an opportunity to do so.

This might look like:

  • Offering a one-paragraph explanation of the purpose behind the assignment in the assignment instructional document, and why the things that might seem tedious or redundant are, in fact, important and important to do without outside help (of course, making sure that your assignments are not needlessly redundant or tedious). You might also include rationale language in a Brightspace announcement or during class time.
  • Demonstrate that you are also learning about A.I. and are trying to figure things out, just as they are. Create expectations about A.I. use together that might become syllabus policy, or part of rubric criteria for an assignment, showing that you want to rely on their critical judgement, too (thereby encouraging them to engage in critical judgement).
  • Making explicit what often remains implicit: that true learning comes out of the relationships we have and nurture with others. Help them (and maybe even yourself!) discover or re-discover the wonderful dynamism and world-changing potential of human beings learning together. For more guidance and frameworks, Indigenous pedagogies and Indigenous conceptions of teaching and learning can be especially helpful.
  • Talking about your own thinking and working processes, or early struggles with assignments typical of your discipline. Explain why you think those early struggles contributed to your learning as a scholar in your field, or as a professional.
  • Creating space to learn a little bit more about your students and then helping them construct their own assessments, research questions, or work plans.
  • Engaging with students more closely in various stages throughout the assignment process, by providing more opportunities to get feedback; employing peer review; or evaluating work together (for example, have the student fill out a self-assessment first, and move from there to a final judgement on the work).

 

Get creative with assessment structure, environments & load

Often, students feel at sea when navigating assignments that have complex instructions or multiple components. Relatedly, we might consider the biases we might have towards typical disciplinary assignments; just because we had to write 5-page research papers in most terms over four years doesn’t mean that’s what should be set as a default assignment. Longer isn’t necessarily better, or most context appropriate.

Consider these ideas:

  • Break down the assignments by scaffoldingmulti-staging; embedding an abundance of ways to get formative feedback along the way and/or including detailed, step-by-step instructions that include estimated time to spend on each step. Removing the stress of completing an entire complex assignment by themselves takes away a key motivator for students getting outside help.
  • Assignment end-products, or parts of the assignment process, might be better matched with a media other than writing. A one-paragraph research topic assignment could be verbally delivered during office hours, or via audio clip on Brightspace.
  • Create collaborative and communal assessment environments, bringing what is usually done alone and outside of the classroom back into the classroom, while reducing assessment stress. Writing circles and collaborative final exams (see pages 94 and 102 for two great examples) can help.
  • Reduce the overall assessment burden, breaking away from the assumption that more pages written equals more learning done. As much intellectual work might go into preparing to deliver a “3-minute Thesis” as writing several pages of a research essay. And it probably exercises more skills, too.

 

Emphasize process over product

Most instructors most likely include verbs such as “interpret,” “distinguish,” “critique,” “evaluate,” and “investigate” in their learning outcomes. The road to those intellectual actions is rocky and winding; students need to brainstorm, figure out how, play, edit, revise, start over, collaborate, discuss, debate, and have their minds changed. So, why do we so often solely require, and solely reward, a polished essay, prioritizing product over process?

Here are some ways to do the reverse:

  • Assess their contributions to a socially annotated text.
  • Spend time discussing the benefits of marginalia and have them turn in their own (screenshots, scans of pages; concept maps, etc.). Similarly, students might turn in their notes, concept maps, and other artifacts of thinking and learning.
  • Use labour-based grading schemes—in other words, grade for quality of effort, not for quality of the product.
  • Ask for meditations on their own thinking process and work process—these could be metacognitive reflections on how they approached the assignment or personal reflections.
  • Use Word comment and track changes functions to review student work and then ask them to respond to those or write a new draft in response to the comments.

 

Incorporate a critical A.I. lens

Students need the time and support to critically examine and explore new technological tools and imagine the implications (positive and negative) for their lives and the lives of those that belong in their communities. They need time to develop their own ethical stances.

Here are some ideas:

  • Co-create syllabus language, assignment instructions and rubrics together.
  • Set assignment prompts that help students explore the implications of A.I. use for the discipline, their lives, or their learning. Include readings in your course that discuss the pros and cons that students can draw upon in their explorations.
  • Test out the A.I. together—compare human and A.I. output (a piece of creative writing, a laboratory experiment) to understand the limits of the A.I. (the instructor’s account should be used to generate output, given data security and privacy concerns)
  • Students cannot be required to use A.I. or create accounts for A.I. platforms; though, for those interested students, you might suggest a voluntary assignment that asks students to compare and reflect on their learning experiences between attempting work with and without A.I. Suggest to students that they create an email account especially for signing up to these platforms or use your instructor account to generate a batch of outputs.