AI has ushered in a new era in education, presenting both opportunities and challenges for teaching, learning, and assessment. It offers immense potential to personalise learning experiences, optimise teaching strategies, and streamline assessment processes. However, the integration of generative AI in education also raises concerns about privacy, potential data biases and impact on assessment integrity. You can get a quick flavour of AI in Martin Compton’s post on AI: Friend or Foe?
At UCL there is an AI Expert group who are who are “working to provide practical solutions and guidance following the emergence of AI in education and assessment“. The group are considering areas including AI academic skills, regulations and assessment redesign. Bringing AI in to assessments is one area of interest and the group intend to support academics who are interested in leveraging AI extensively within their assessments.
In the following video case study (12 minutes 43 seconds) Dr Josep Grau-Bove, Associate Professor in the Bartlett School of Energy Environment and Resources talks about how his students are using AI in his module Machine Learning for Heritage (BENV0115).
What we did and why
Dr Josep Grau-Bove leads the MSC in data science for cultural heritage and has been encouraging the students on the course to use AI in their assignments. The assessment for the module is a report on a machine learning project and this year has been transformed to incorporate AI more strongly. The students are provided with a data set and have to work through a classification problem e.g. they are giving given images of paintings by certain artists and need to develop a machine learning algorithm that classifies them according to the artist or identifies forgeries. They have been encouraged to use tools such as Chat GPT or equivalent Large Language Models (LLM) to either write part of the report, or part of the code. When they submit the assignment they need to indicate the contribution of the AI tool, the prompts used and reflect on their experience.
As Josep explains: “I was interested in their opinion because my Master’s students are really bright. I know they are passionate about machine learning. And through these personal reflections, I wanted to see if they found it useful, if it helped their learning, or if they actually thought it was good for programming. I got a great diversity of responses.”
Josep paired this assessment with a small data collection exercise in which he shared a survey with the students before and after the assignment asking them about their attitude towards generative AI. “The main trend that I observed is that before experimenting with language models to help them with their assignment, their views were quite polarized, almost divided 50/50 among those who believed AI was so powerful that would replace programmers and all students would benefit from it in their assignment and those who are really sceptical. After using it in their assignment, the views became much more cantered.” The students became much better informed and recognise the opportunities and challenges generative AI brings.
Josep was initially concerned that there would be students who would not want to experiment with the tools but found that even the most skeptical found the exercise useful. He was also worried that some students would view this an advocation of ‘cheating’ but in reality the exercise demonstrated that “their human brain still can overpower these tools for the very specific problem we were tacking“.
Joseph’s tips for those who would like to use AI in an assessment
1. Do not be afraid to integrate AI in to your assessments
2. Learn from how the students use generative AI in their work – in a natural and agile manner. They adapt it in to their workflow.
3. Try bringing AI into conversations and consider using it earlier in formative assessment so you can discuss it.
4. Consider having a week where you and your students are free to experiment.
You can see all our other posts on Artificial Intelligence.