Submitty is an open source course management and assignment submission tool developed by the Rensselaer Center for Open Source Software (RCOS) in the Department of Computer Science at Rensselaer Polytechnic Institute. It is designed to streamline the administrative and grading processes associated with assessments, particularly in computer science and related disciplines. Its key features are: assignment submission, grading interface and automatic testing of code assignments i.e. instructors can define test cases and rubrics to evaluate student code automatically. Submitty is available from GitHub.
At UCL it is used on courses taught as part of the Advanced Research Computing Centre, UCL’s centre for infrastructure and innovation in digital research. Currently Submitty is used on Research Software Engineering with Python (COMP0233) for testing submissions and marking, and on Research Computing with C++ (COMP0210) for marking only.
In the following video case study (13 minutes 3 seconds) David Pérez-Suárez, Principal Research Software Developer, tells us why he started using Submitty as part of the submission process for assessments, and the impact this has had on the student assessment experience.
Overview
David Pérez-Suárez and his team have moved away from traditional written exams in their programming courses, recognising that they often fail to align with the course’s objectives. Instead, they have transitioned to using coursework and assignments, most of which involve coding. This shift allows them to automate parts of the assessment process, providing valuable insights into student performance.
One of the key features of Submitty tool is the ability to allow students to submit and test their code before the assignment deadline. This pre-submission testing serves a dual purpose. Firstly, it helps students identify issues and areas for improvement in their code, earning them essential early feedback. The students test their code for specific components of the marking scheme, the process also helps eliminate common issues, such as incorrect file naming conventions. Secondly, it aids UCL’s teaching staff in assessing the submitted code more efficiently. This early testing phase ensures that when the final assessment is conducted it can be processed without encountering small problems that could significantly affect the final grades.
Submitty facilitates collaborative grading by allowing multiple instructors to evaluate assignments simultaneously. This collaborative approach enhances efficiency, as instructors can divide the workload and cross-check each other’s assessments. Submitty also records the changes made during grading, providing a transparent audit trail.
It has helped them (the students) to get a better submission and it has helped us (the staff) to get a better grading experience
While Submitty has been a valuable addition to UCL’s programming courses, it does present some challenges. The most significant is the need to write effective tests that accurately assess students’ skills and this be time-consuming. Additionally, the existing system’s capacity to handle a large number of submissions simultaneously is an area that UCL aims to improve. David is also exploring ways to prioritise students who submit their code for testing earlier, and looking ahead he plans to promote the use of Submitty more widely across the institution.
If you are interested in using Submitty in your teaching or assessment contact arc-teaching@ucl.ac.uk for further details.