Yesterday was the annual UCL Education Conference with a theme of Designing Futures-focused Education. The event was jampacked with a pre-event online session, multiple asynchronous sessions and a day of in-person keynotes and parallel sessions.
The entire Digital Assessment Team managed to attend at least some of the event. Here, wearing our ‘assessment glasses’, are some of the choicest outputs.
Welcome address
UCL President and Provost, Dr Michael Spence opened by asking us to think about how we can create an educational environment that prepares out students for the future? How can wesupport them in being the people who “tell the machines what to do rather than the people whose job is replaced by machines?”
His suggestions are:
- To offer them a deep grounding in a discipline.
- To encourage them to be able to step out of that discipline. The future will be require T shaped jobs and the ability to work in multi-disciplinary teams and speak the language of other disciplines.
- To garner an international perspective.
- To ensure the capacity to acquire synthetic intelligence. Students require experience of real world problem solving, where there are no known solutions.
Ongoing UCL work on the Curriculum architecture project, support for campus-based education and exceptional student leadership will help build this educational environment.
Keynote on Generative AI and Education Futures
As organiser Jesper Hansen, Lecturer (Teaching), UCL Arena Centre pointed out, AI was barely on most peoples’ radar at last year’s conference (see my write up). This year it would feel odd to not acknowledge the huge shift AI is having and will continue to have on education. The opening keynote was given by Professor Mike Sharples from the Open University in conversation with Mary McHarg, SU Activities and Engagement Sabbatical Officer.
After a quick overview of the affordances and limitations of generative AI Mike reflected on the continuum of development in dealing with AI. This moves from:
- Banning – this approach has been taken by some countries and institutions but seems set to fail. Confident students will continue to use AI whilst those less able are left feeling anxious and concerned.
- Evading – Invigilated exams are both costly and limited, and asking students to reference when using AI is going to become increasingly difficult once tools like co-pilot become mainstream.
- Adapting – Will require new methods of assessment, new guidance and policies and building AI literacy.
- Embracing – Is likely to take a considerable amount of time and resource from all.
The discussion with Mary initially built on the emerging theme of a new divide opening between students who can afford and know how to use these tools, and those that can’t. How can we make these tools accessible to all students? How can we build in AI literacy to our degrees? How can we ensure the data used by the tools is openly available and understood? Are open-source tools like BLOOM able to help? BLOOM generates text in multiple languages and is transparent about the data used. Mike explained that to some extent the bigger the dataset the less trustworthy the AI tool will be because it pulls in the biases from all over the Internet. Cultural biases are particularly susceptible. For example much of the regulation so far has come from Europe (Italy has banned Chat-GPT) and the US, while parts of Asia (for example Singapore) may take a different approach. What we need, and are starting to see, are for AI tools to be trained on curated expert data sets, for example in finance or biomedical sciences. We need to ensure universities are involved in the creation of these datasets. Currently much of the running on AI is being carried out by start-ups, and while they don’t necessarily come from a bad place they do have their own agenda, but there are real opportunities for universities to lead
The discussion then moved on to the impact AI will have on assessment. Mike spoke positively about the opportunities for building in creativity and project work, and for new ways of textual analysis and divergent thinking. Despite being on the Turnitin advisory group Mike warned of the false positives (2-5%) AI detection tools bring and how ‘AI will win’ in this arena. The open letter on halting AI development has to some extent confused what is happening now (tools like Chat-GPT) with Artificial General Intelligence, which remains a long way off. Apparently OpenAI won’t produce GPT-5 for the time being and instead will refine their current tool.
Other questions from the audience asked how secure is Chat GPT in terms of the data which is inputted? Apparently Chat GPT have agreed not to store conversational data but this may not be the case for all tools. Tools like AutoGPT, which have the ability to “self-prompt” to achieve any given goal, do have a persistent memory. The conversation was truly fascinating and excellently facilitated by Mary.
Panel on HE challenges
The afternoon kicked off with a panel chaired by Pro Vice-Provost (Education – Student Academic Experience) Professor Parama Chaudhury. She was joined by Emma McCoy, Vice President and Pro-Vice-Chancellor for Education at LSE, Professor Susanna Quinsee, Vice-President (Digital and Student Experience) at City, University of London and Professor Adam Habib, Director of SOAS University of London. They explored some of the biggest challenges facing the UK HE sector today including commercialisation (Cambridge professor Stefan Collini recently warned that universities will soon become third class enterprises); finances and increasing expectations from students. They explored the effect the pandemic has had in reality and with regard to perceptions of online learning (“What do students gain that isn’t just content? The value is in interactions”). And explored how many of the answers lie in the intersection between global and international (glocal) and the interception between different disciplines (inter-disciplinary). Emma explained how the LSE100 course, a flagship interdisciplinary course for all undergraduate students, had tried to look at these big questions.
Education at UCL
The main stage activities were completed with a conversation between Vice-Provost (Education & Student Experience) Professor Kathy Armour and Hamza Ahmed, SU Education Sabbatical Officer. Kathy explained how the new UCL Strategic Plan 2022-27 and education framework are crystallising the areas of importance for students. They want their educational experience to be research informed, rewarding, values-led and collaborative. The pandemic and recent AI challenges have only shone a light on existing issues and we now find ourselves in an ‘existential moment in education’, with the opportunity to make real change.
Parallel sessions
Throughout the day there were parallel sessions comprising of cases studies and individual presentations. Some of our favourites were:
- Paul McFarlane from Engineering presented on how student choice in a Criminology formative assessment has been used to motivate students and foster a sense of ownership in the learning and assessment process. He explained how much of their work was grounded in supporting mental health, The assessment was 100% portfolio and students were offered three modes of assessment (essay, briefing report and video) and then asked to chose a topic to cover, There was a 60-40 split between the and the report and the essay. The team plans to bring more choice in to the module in the future.
- Evi Katsapi, IOE presented on a new assessment approach in Psychology and Neuroscience in Education (PNE). Originally 5 pieces of group-work component, the PNE assessment was re-designed to include: 1) a 5-minute VLOG and 2) a full research proposal. Students were actively involved in the process and Evi and team began the conversation on the assessment design by talking about UCL’s pillars of employability and transferrable skills. Both assessments were well-scaffolded. For the research proposal the module leaders shared their own (failed) grant proposal and for the VLOG the PGTA created a sample VLOG and presented to the students while sharing what they found difficult.
- Eliot Hoving (previously a Digital Assessment Advisor), Isobel Bowditch (a current Digital Assessment Advisor), Leo Havemann (Arena) and student and Leshi Feng shared findings from the workshops on academic integrity. While there were shared concerns around student anxiety driving misconduct, students tended to refer to ‘referencing’ (as a proxy for plagiarism) while staff highlighted contract (and AI) cheating and collusion as major areas of concern. We will have a follow-up report on the work.
- Martin Compton (Arena), Dr. Cathy Elliot (Vice Dean Education SHS), Dr. Fleur Corbett (Div of Psych and Lang Sciences), Dr. Jennifer Egbunike (Health Policy and Organisational Studies), Connie Gillies (3rd Year Politics and IR student), Dr Rebecca Lindner (Arena) and Marieke Guy (Digital Assessment) took part in a panel session on Is it really impossible? Exploring ways we can ‘free the feedback’ and reconnect students with the processes of their education and challenge obsessions with grades’. The session explored some of the ideas presented by Cathy Elliot in the video case study on Reducing the emphasis on grades and was on behalf of the Freedom to Learn movement (internal only).
What a turnout!! UCL is VERY interested in #FreedomToLearn #Ungrading fantastic round table at #UCLEducationConference with @DrCathyElliott Connie Gillies @mariekeguy Fleur Corbett, Jennifer Egbunike and Becky Lindner pic.twitter.com/ejUEpn9V9E
— MartinCompton (@mart_compton) April 19, 2023
Asynchronous sessions
There are three asynchronous sessions on assessment available on the UCL Education Conference blog including an overview of the Lockdown browser. Also embedded below.
- Is AI impact on assessment an opportunity to redevelop inclusive oral examinations in medical and biological sciences? – Sandra Martelli
- Approaching exams and building resilience – Dr. Valentina Amuso
- The UCL lockdown browser pilot – Marieke Guy
