Generative AI technologies, such as ChatGPT, have raised concerns regarding academic integrity and the potential use of generative AI applications for student coursework output. Conversely, generative AI tools may have several positive and beneficial applications for student learning and for incorporating into existing student assessment models.
As AI technologies become common in the workplace, including across the science and medical professions, universities have a responsibility to train their students on their proper use, capabilities and limitations.
This project firstly involved an evaluation of year 1 and year 2 module coursework assessment on the BSc Applied Medical Sciences undergraduate programme as part of a UCL Co-Creator: AI Project. The aim was to evaluate:
- The potential impact on academic integrity of ChatGPT on existing coursework assessments.
- Existing assessments that may be appropriate for the integration of ChatGPT or other AI tools.
- The key challenges that student and staff may face regarding ChatGPT.
The following video is an edited version of my conversation with my student co-creator (Ellie Ingle).
The following is a summary of our key findings:
- The standard assessment paradigm (MCQs, SAQs, essays) may be particularly susceptible to ChatGPT generated output.
- ChatGPT displays inherent bias in some generated answers.
- ChatGPT is poor at accessing and evaluating the scientific literature.
- Training on generative AI is required for both students and staff.
- Important to ensure students have equal access to AI tools.
- Embedding generative AI technologies into module curricula and teaching practices could benefit student learning.
Further details on the student co-creators project can be found using this link: https://sway.cloud.microsoft/9d5Iz97xVF8ztZPt?ref=Link&loc=mysways
It is interesting that equity and ethical considerations, both in terms of access and technical ability (the digital divide), were main themes. In a subsequent conversation with ChatGPT, this is what the generative AI had to say:
The second primary project was associated with a year 2 undergraduate module, Molecular Basis of Disease, and the re-design of its written coursework assessment that was redesigned to incorporate an element of ChatGPT generated output. The coursework was designed to fulfil two main objectives.
- Provide students with an opportunity to explore the capabilities and limitations of generative AI.
- Provide an authentic assessment where students can use their critical evaluation skills.
The coursework consisted of: “Critically evaluate the grant proposal generated by ChatGPT.” The grant was based on the vasculopathy associated with scleroderma and the students were assessed on their critical evaluation of the scientific accuracy, rationale, experimental design and patient benefit, as outlined by ChatGPT in the generated proposal.
Several opportunities for formative feedback were provided throughout the, further embedding generative AI into the syllabus.
Thirdly, it was important to gain an understanding of the student perspective and experience in using generative AI in their learning and assessments. We used a combination of a student questionnaire and a focus group to provide student insights into AI technologies, with broad applications and interest across the higher education sector globally.
Analysis of the findings from the research project are ongoing, however, the following themes arose from the positive student responses with regards to using generative AI for coursework and university studies.
- Time-saving
- Clarification and understanding
- Grammar and language improvement
- Idea generation and summarization
- Research assistance
- Assessment and coursework assistance
- Language assistance
Students had further concerns about he the use of generative AI coursework assessments at university, including the following themes.
- Accuracy
- Misleading
- Plagiarism/Cheating
- Brain dead
- Lack of recent information
- Inaccurate data/information
Designing new coursework assessments or redesigning existing assessments to incorporate generative AI technology will require consideration and time. This is an open-ended project and AI technologies are advancing rapidly. Hopefully our project will foster further discussions on using AI in your assessments, teaching and learning practices.