While UCL has committed to an assessment platform (Wiseflow from Uniwise) we continue to explore how our assessments will be delivered, using this platform and possibly other platforms. One area we are interested in is Bring Your Own Device (BYOD) exams in a face-to-face setting using a lockdown browser.
BYOD have been run at Brunel using Wiseflow since pre-covid. Over the last few months we have been talking to the Brunel team with a view to carry out something similar at UCL. You can view an ALT case study on the Brunel experience and see related guidance from their website.
At the end of August we were able to test out the Wiseflow lockdown browser (Flowlock) using a small group of students. As the Wiseflow site explains, this browser uses “enhanced security measures to support supervised closed-book testing, either on-site or remotely. The lockdown browser is a small application that provides the necessary tools to complete your exam but restricts your computer from accessing unauthorised material during the test period.“
UCL has made a commitment to not enable proctoring on online remote assessment. The lockdown browser will be used in face-to-face assessments with a human invigilator only.
In order to run a mini-pilot testing the browser on a low stakes assessment we utilised a small group of postgraduate Pharmacy students who were carrying out research activities over the summer and therefore available on site. The students were invited along to a session in which they would complete an exam that had been already used earlier in the year and then participate in a focus group asking for their feedback on the experience. The students were paid for their time.
Preparation
Prior to the pilot the students were provided with guidance on how to download the lockdown browser is available from the WISEflow support page. The students were asked to ensure that their laptop was fully charged and had all the latest updates installed for the Operating System and any anti-virus software required. We also asked them to ensure they could connect to Eduroam (on campus) and that their password did not require updating. Those without a suitable laptop were asked to contact us in advance.
During the examination
On the day, once students had arrived and were seated at the desk with their name we gave a short briefing introducing ourselves and outlining how the pilot would run.
Participant password on main screen
This covered:
- a quick overview of the FLOWlock browser and the login process
- timings for the session
- an explanation of the participant password (used to start the exam) and invigilator password (used to submit the exam and for technical issues)
- requirements for the exam – what we wanted the students to test out (e.g. the drawing and appendix features, permitted/whitelisted sites)
- general examination regulations (e.g. remaining silent, turning off phones) – while this wasn’t a real exam we wanted to replicate the exam environment as much as possible without causing too much stress
- how to get help if there were any issues
The examination ran for around 50 minutes during which time attending staff carried out various roles. We had two invigilators who walked round the room supporting staff with technical queries. One manager monitoring the participation overview online. One online invigilator monitoring the participation overview online. Other staff taking notes and recording outcomes.
After the examination
After the students had submitted their assessment in the lockdown browser we ran a short focus group session looking asking student what they thought of the following areas
- Guidance given prior to exam
- Downloading the lockdown browser – what type of device do they have> What OS?
- Functionality
- Accessing the flow
- Top level navigation and functions – countdown clock, battery, messages, flow description
- Zooming and any accessibility issues, keyboard short cuts
- Inserting symbols
- Copy and pasting
- Main text editor
- Permitted internet resources
- Right hand side bar options
- Overview
- Assignment
- Table of contents
- Appendix material – adding in images and drawings
- Revisions
- Going offline
- Submitting including offline upload
- Comparison with controlled conditioned exam at home or F2F? And any other assessment experiences
The students were very open about their experience and shared lots of constructive feedback.
Findings
The session was incredibly helpful. Carrying out the pilot in a low-stakes environment allowed us to test out lots of potential problems and look at areas including:
- Set up – Ideas for room set up, wifi requirements, password issues, standardised equipment (laptop, charger etc), using departmental laptops, buffer window time for login, attendance recording…
- Laptop requirements – browser app set up, mac issues, webcams…
- FlowLock assessment – navigation, layout, appendix manager, practice assessments, whitelisted sites, direct messaging, coversheets,,,
- Auditing – participant logs, offline access…
- Student perspective – concerns and anxieties, need for clear guidance and explanation for use of the lockdown browser, requirement for practice assessment…
We have passed some of this feedback to the platform developers with a view to potential enhancements.
Our next steps will be to scale up and run a pilot on a live assessment. We hope to do this early on in the new academic year.