Generating Random Mastery Quizzes with Nothing But LaTeX

Mastery Based Grading can encourage your students to adopt a growth-mindset mentality, help reduce math anxiety, and give students more control over their own learning. But it also seems like it would take SO MUCH OF YOUR TIME. Mastery Based Grading is at the epicenter of the eternal struggle between teacher and student, learning and test-taking, and figuring out how to do everything in just 24 hours a day.

In this article we’ll discuss how to create a simplified, streamlined MBG environment for Calculus based on randomized multiple-choice Mastery Quizzes with embedded coded answer keys generated from (mostly) simple LaTeX documents.

Growth Mindset

What got us thinking about Mastery Based Grading in the first place was this video from Eduardo Briceno about Carol Dweck’s work on growth mindsets vs. fixed mindsets:

If you don’t have time to watch the video, just look at the diagram below from Ameet Ranadive. If a student has a fixed mindset then they tend to believe they are “just bad at math”, and a bad test grade is cause for shame and sadness. Students with growth mindsets see hard problems as challenges, and failures as opportunities to grow their abilities:

The idea that students should think of their educational journey as an ongoing learning process rather than a judgemental evaluation process is great, but to be perfectly honest, in my usual calculus classes I was using a traditional testing structure that is a judgemental evaluation process. The traditional model of “try to do homework, take Test 1, fail Test 1, then start preparing for Test 2 but hopefully do it right this time” isn’t really a growth mindset model, especially when the material on Test 2 is completely different than the material on Test 1. In contrast, Mastery Based Grading basically encourages you to grade students based on their mastery of identified skills, then allows to students to improve on those skills and get re-evaluated, thereby growing their knowledge in a really tangible way.

What is Mastery Based Grading?

Honestly, I think different people give many different answers to this question, and it’s also mixed up with Standards Based Grading, which is another thing but also kind of the same thing. Although everyone seems to do something different, perhaps the main goal is that students are told what they are being graded on and then given ongoing opportunities to level up.

Here is a really simple example of Standards Based Grading in a K12 setting, from TeacherEase:

And here is a really in-depth Mastery Based Grading Rubric from Great Minds which includes very specific standards and very specific mastery levels for those standards:

This job is too hard

At this point I’m starting to panic a little bit. I can’t possibly track this kind of information for every problem, and in fact it’s going to be really difficult even just to break all of calculus down into bite-sized standards that I can then evaluate for mastery and then let students do again and then evaluate again, and really this sounds like a lot more grading, doesn’t it? I don’t think I can do this if it involves more grading.

This is a put-your-own-mask-on-first moment. I need to protect myself and make sure that I’m okay and will survive teaching calculus before I can be a good teacher to my students and make sure that they survive learning calculus.

During my brief time working in industry I sometimes heard people talk about the “80-20 Rule”, which says that you can usually find a way to get 80% of what you want for just 20% of the work it would take to get all of what you want — meaning that the last 20% of your project goal can take 80% of the work. The moral of the story is that you might want to re-evaluate your project goals and ask yourself if 80% of them is actually all you really need.

It’s just a gimmick that in this rule you are pondering the numbers 80/20 for your goals and then the same numbers 20/80 for your work, but this conflation does lead to some of the absolute worst graphs I have ever seen on the internet. Math people, if you ever want a good laugh about terrible pie charts, just Google the 80/20 rule to see gems like these:

    

    

MBG without OMG

Here are the five simplified-MBG principles I followed in order to keep my sanity while attempting to start using Mastery Based Grading:

  1. Instead of standards, use the sections of the book.
    It’s probably best to list explicit calculus content standards and then key all your exam and homework problems to those standards, but I figured I could get 80% of the benefit by just using the sections that the book was already divided into. …It’s one of my favorite books, so probably split up how I like it :)
  2. Have a very controlled retake policy built into class time.
    I know from giving various types of Gateway Tests in the past that students can sometimes procrastinate and/or abuse retake policies, and I wanted to have control over that time and process, so I gave assessments only during actual class periods, and at specifically scheduled times.
  3. Make all assessments multiple choice.
    If I was going to let students retake assessments potentially multiple times each, then those assessments absolutely had to be easy to grade. Over the years I think I’ve gotten pretty good at writing multiple choice problems that are actually meaningful, so I decided to again get 80% of what I wanted and make all of the assessments multiple choice.
  4. Avoid having to keep track of versions and keys.
    I most definitely did not want to have to keep track of which students had taken which versions of which quizzes, or to keep multiple keys around for each version of each assessment. To get around this I leaned heavily on my friend randomness, and built each assessment by choosing randomly from a large test bank of problems. I also embedded codes into each assessment so that I could always recover the key from the assessment paper itself!
  5. Figure out how to generate randomized quizzes in LaTeX.
    The key to making the points above happen was to find a way to get LaTeX to generate as many random multiple choice quizzes as I wanted for each section of the book. I know you can do this in Python and then export to typeset in LaTeX, but I wanted to make it happen with only LaTeX. Why? Both so I could just use LaTeX files to generate quizzes, and so that I could share the files with other people; nearly every mathematician knows how to use LaTeX, but not everyone knows how to use Python!

I want to stress that other people are implementing Mastery/Standards Based Grading in a way that is much more comprehensive and thought-out, with much care given to choosing standards and providing quality assessments. I wish I could be more like those people. But for now I am pretty happy with what I’ve worked out, and will be using it in my calculus class this coming semester for the third time.

Lazyman’s MBG in Action

In my calculus-with-precalculus class this spring, students’ grades will be based entirely on their performance on 18 Mastery Quizzes, one for each section of the book we’ll be covering. Students will take these Mastery Quizzes in class according to a schedule, and will also take 10 Retakes of their choice during class periods. In addition they will have four “Group Retake” opportunities (this is new — I’ll let you know how it goes after I find out). Their Final Exams will be choose-your-own-adventure style, where each student can choose to take any 8 Retakes.

For details see the course website, specifically the Syllabus, Homework Calendar, Mastery Quiz FAQ, and Grades FAQ.

At the end of the semester I’ll look at each student’s highest grade on each of the 18 Mastery Quizzes and then determine their course grade as follows:

  • To earn a grade of “A”
    • 3 points on at least 15 of the 18 Mastery Quizzes
    • No Mastery Quiz scores below 2
  • To earn a grade of “B” or higher
    • 3 points on at least 9 of the 18 Mastery Quizzes
    • No Mastery Quiz scores below 2
  • To earn a grade of “C” or higher
    • 3 points on at least 6 of the 18 Mastery Quizzes
    • No more than 3 Mastery Quiz Scores of 1
    • No Mastery Quiz scores below 1
  • To earn a grade of “D” or higher
    • 2 points on at least 9 of the 18 Mastery Quizzes
    • No Mastery Quiz scores below 1

Schematically, that boils down to the following table, where each checkmark represents a score of 0, 1, 2, or 3 on a Mastery Quiz, and the columns show what you would have to do to get into each course grade range. The rightmost column is for the students to put in their own checkmarks so they can see how they are doing:

Of course there are lots of situations that don’t fit into this chart, but this is the aspirational guide for grades of A, B, C, and D. It turns out to be not so difficult to assign grades based on this chart; you just have to assign A-, B+, etc, depending on how close students are to the main grade columns. (Much thanks to Chris Hanusa, who helped me puzzle out how this grading scheme should work!)

Here’s what things looked like near the end of the semester, just after one of the last Retake opportunities. Scores in red indicate which quizzes were chosen by each student. You can see a lot of students finally getting scores of “3” after one or two retakes, but also students who aren’t moving their grades very much. Guess which ones did the homework!

Generating Random Mastery Quizzes with Embedded Keys in LaTeX

Now let’s finally get to the thing that makes all of this possible: using LaTeX to generate random quizzes with embedded answer keys! Without this piece of the puzzle none of the simplifications above would have been possible.

We constructed multiple-choice test banks based on the homework assignments for each section in the book. That way, students know exactly what to study if they want to take (or retake) a Mastery Quiz; they just do the homework assignment for that section (ahem). Here’s what two of those multiple choice questions looked like last semester, when we were first learning about the definition of derivative:

At the bottom of each three-question multiple choice quiz is a code that tells me (but not the students) the answers to the problems, as well as whether or not those problems were used on previous assessments, and the relative difficulty of the quiz. The code has a lot of distractors in it, so it’s pretty impossible to break without inside knowledge, but I also change how it works periodically just in case. If you’re clever about how to construct the code, you can even build in some error-checking. Below the code we reprint the homework assignment, to remind students that to study for any future retakes they need to look at the entire assignment again, not just the problems they missed on the quiz:

This is what it looks like when generating quizzes in LaTeX. Each time you compile, LaTeX will give you a different random quiz. You can also change multiple choice orderings and whether or not answers are visible (for proofreading). In the screenshot below, there happens to be a problem from the test bank that got chosen twice, which obviously is not good! I didn’t know how to avoid this problem in LaTeX, but I could detect when it happened, so I added a warning note at the bottom that appears whenever there are duplicate problems on the quiz.

Using the LaTeX above you just compile until there isn’t a warning (and you’re happy with difficulty level, etc), then print one copy of the quiz. Recompile and print another copy. Repeat until you have enough copies; everyone in the class is taking a totally different quiz, but on the same material. It’s actually very fast to grade these, even though they are different for each student. With 32 students in the class I can grade the entire stack in less than 15 minutes.

Student Survey Results

So what did the students think? As usual, results vary from student to student. Some students found that the consistency and predictability and continuous low-stakes nature of the Mastery Quizzes reduced their anxiety significantly. Others would have preferred to have the standard “three tests and a final” format. Overall I’m not sure students got higher scores, or whether or not they learned more… but the psychology of the class was markedly different. Everything always felt like “I can’t do this… yet“, instead of “I can’t do this”. Students knew they could improve any bad scores, and they knew exactly how to do it. And the Retake system kept students focused on the material they didn’t know how to do (yet!).

Here’s some data taken both in the middle of the semster and right before the end:

Why is this post so long, I just wanted some LaTeX files!

If you want to try this yourself, you can download the LaTeX files here:

You’ll see that there are options for generating “first”, “random”, and “last” quizzes. That’s because the first time students take this quiz as a class, so they all take the same quiz (with questions and answer choices permuted, but the same three questions). After that all retakes are random, and on the final I generate questions from a separate test bank so that students can’t just collect previous quizzes to study for the test.

Please note that I’ve replaced my own rather complicated answer key code with something very, very stupid in this example; be sure to change the way the code is generated at the end of the Generator file before you use these files!

If you have any questions about using these files, please feel free to ask me! I’d love to hear if these files are useful to you, and about how you are implementing MBG in your own classroom. Good luck everyone and have a great semester :)

 

——————

As an Amazon Associate we earn from qualifying purchases, so if you’ve got something you need to pick up anyway, going to Amazon through this link will help us keep Hacktastic running. Thanks! :)


Leave a Reply

Your email address will not be published. Required fields are marked *