This Dagstuhl Seminar will identify challenges and potential solutions for the practice of mastery learning within tertiary computing education in the context of the recent rapid advancement of highly scalable large language models (LLM). While an understanding of the capabilities and weaknesses of LLMs allows for its guarded use by academics, its use by students warrants in-depth attention. When considering the need for repeated practice within mastery learning and the lack of in-depth knowledge specific to novice programmers, it is critical to understand how to effectively incorporate LLMs into the mastery learning process.

Mastery learning is an educational approach where students master a specific set of skills before being allowed to move on to a different set of skills, often more advanced than the previous. A differentiating aspect of mastery learning is that students are only allowed to move to the next set of skills after they have demonstrated that they are proficient with the current set. Students are given ample opportunities to practice, to receive timely, supportive feedback, and to demonstrate mastery. Instructors using a mastery approach must have a deep understanding of student progress and structure support accordingly.

Many computer science topics, including programming, appear to naturally lend themselves to a mastery learning approach. However, adoption of mastery learning is limited beyond the first year of tertiary education, where traditionally students learn how to program. A typical implementation of mastery learning in a programming course features a pass/fail assessment based on a sequence of several mastery tests.

The wide availability of LLMs provides both opportunities and challenges for mastery learning. For example, the duality of the mark, the pressure of the tests, and the availability of LLMs can lead students to overreliance for both simple and complex problems, and in some cases, plagiarism and collusion. However, LLMs may also be used to provide timely feedback, if framed appropriately, or to create new resources for practice, which would reduce the cost of adopting a mastery learning approach.

In light of this, our questions are:

• What learning theories and pedagogies support implementation of mastery learning in the era of LLMs?
• To what extent can we co-implement mastery learning with AI? If so, how can we use LLMs to encourage adoption of mastery learning more broadly, beyond CS1 and across all years of a degree?
• How can we implement mastery learning at scale?
• How can mastery learning be designed to reduce the risk of plagiarism?
• How can LLM-based pedagogical tools be designed to support mastery learning?

The timeliness of this seminar is critical: in the last twelve months, LLMs have revolutionized many commercial fields and their use in education continues to grow. This use is mostly by students who use LLMs to generate answers to school work, while universities are grappling with the impact of LLMs on assessment policies and pedagogy.

Creative Commons BY 4.0

Nick Falkner, Juho Leinonen, Miranda C. Parker, Andrew Petersen, and Claudia Szabo