Tag Archives: AI

AI: Shorthand and eponyms

Date
Posted by
Comments1

One of the things we do badly and get agitated about a lot is the naming of things to do with technology; not least in the realm of digital education. Just ask a room of HE folk for a definition of ‘Blended’ and ‘Hybrid’ and wait for saloon brawl to ensue. So it is with some of the language that is emerging as default in relation to all things ‘Artifical Intelligence’, notably and especially the large language model (LLM) ‘ChatGPT’.

Shorthand

I do understand when folk get antsy when something is called a thing that it isn’t, isn’t exactly or isn’t entirely. But, unless you’re a specialist, a pedant, or willfully awkward (and granted a lot of academics are at least two of these- no offence intended), we may as well get on with saying strawBERRIES, peaNUTS and TIN foil even if they are no such thing. In that vein, I am more than happy to use ‘AI’ as a shorthand for all the stuff that is vexing, perplexing and flexing educators just now. I’m afraid that if someone starts saying: ‘Well technically, it’s not artificial intelligence per se, but rather a fine-tuned large language model…’ I can feel my eyes glazing over. Obviously this stuff is fundamental if you are a computer scientist but the reason such shorthands exist is that they are short, handy (clues are in the word) and suitable for lay users. Experts that know this are one step closer to communicating beyond their discipline.

a mechanical strawberry as imagined with midjourney AI image from text

Eponym

I am much less comfortable with some brand eponyms; especially when the tools and the words to describe them are still evolving. Against my better judgement, I ‘Google’ stuff even if I’m in Safari and I use a ‘hoover’ (even though the only actual- and long discarded- Hoover I ever owned metaphorically ‘sucked’ at the literal thing it was supposed to do). But I am pushing back at the generic use of ‘ChatGPT‘ (OpenAI’s LLM) to refer to the latest iteration of large language models. Chatbots have been around for years and the underpinning technology has evolved rather than suddenly appeared but the genius of the release and the subsequent explosion in use and interest is in the non-techy, almost friendly user interface combined with the now famous fluency and (superfically at least) convicing outputs. The ‘GPT’ part stands for ‘Generative Pre-Trained Transformer’ which certainly needs unpicking to understand but 100 million users in two months is testament to the appeal of this iteration of this particular tool with this particular interface that has led to so many using the ‘ChatGPT’ eponymically. But the ongoing interests in Open AI from Bond villan Elon Musk; the environmental and ethical costs and implications along with the ‘Oh my god ChatGPT could pass an MBA’ educational panic in some quarters mean we could rue generalising the term. Subscriptions to ChatGPT will necessarily change our realtionship with it and, if even half of what reading about Microsoft integrations are to be believed (as well as Google’s “Look at us, we have ‘Bard'”counter-panic) the technology will be no more separate from the tools we use every day than a spellchecker.

My suggestion

All I’m saying really is that we should think carefully about the terms we use now lest they become fossilised, awkward and, worse, provide advertising and subtle condonement to what is just one product amongst many (and with not inconsiderable ethical baggage). So, for my tuppence worth: I think it’s OK for educators and students to use AI as a catch all term for these new dialogic, chat interface tools as well as other generative tools such as Dall-e and Midjourney ‘image from text’ generators and other comparable AIs such as AI music generation and AI video generators. The common denominator is ‘generation’ so I wonder whether we might usefully agree to use ‘AI text generators’, ‘AI image generators’ etc.. . as the default? I have been using ‘language models’, ‘large language models’ and even LLMs and realise that experts would likely prefer this framing but to a lay ear these say nothing about what these tools do and, anyway, when ‘The One Show’ (Popular BBC early evening magazine programme (starts at 22:56) is using ‘ChatGPT’ generically, a technical preference has probably got no chance.

AI and implications for HE assessment – my tuppence worth*

Date
Posted by
Comments1

ice crystals for decoration only

When I can, Friday afternoons are a good time to fiddle with things. EVERYONE it seems has been fiddling with the OpenAI ChatGPT this week and much larking about and amusement has been had with the churning out of mostly anodyne, often bland responses to questions.  Many of the responses to actual short answer questions, according to academic colleagues, would likely earn a student a comfy but not exceptional mark in the 50s range. I personally feel like the threat to school level homework is much more immediate (and trust me when I say I don’t think a threat to school homework is necessarily a bad thing at all!) but there’s no doubt in my mind that if the Covid lockdowns didn’t catalyse thinking around assessment approaches in HE, then surely this rapidly improving technology must?

Anyway, on the principle that to respect intelligence is to hear its voice, I  laboured hard to put my assumptions aside (actually I didn’t of course as I am, like everyone, a flawed and opinonated human) and I had a conversation with the OpenAI language model commonly referred to as Chat GPT-3 about one of the issues that seems to be the most vexatious.  Make of it what you will!  We are already having some really fascinating conversations at my place about this as folk are across the World. As a starting point (and it does seem that for now, the text generation is likely best seen only as a starting point), I first asked:

  1. Martin: What is your name?

 Catherine: I’m sorry, but I am not a person with a name. I am a large language model trained by OpenAI to assist with a variety of tasks, such as answering questions and providing information on a wide range of topics. I don’t have my own personal identity or consciousness. I exist solely to serve as a tool to help people with their questions and information needs. Is there anything else I can help you with?

So, that told me. Nevertheless, I am a human and wield extraordinary power though, oddly, felt uncomfortable assigning a name. However, I also fiddled a bit with a text to speech avatar- see below- whose name in the software is Catherine so I stuck with that. Why did I want a name? Because 1. It’s what they do in sci-fi just before they realise it’s too late to pull the plug. 2. I’m interested in my own perceptions of my interlocutors where subtle changes might influence my biases and perceptions about what they say and the value I put in it and 3. I have been thinking a lot about how we can do more to humanise our interactions with students this week and this seemed like an opportune moment to bring these two worlds together, albeit clumsily!

 2. Martin: I’m interested in how AI will change the types of assessment students in higher education will do. What are your thoughts?

Catherine: As AI technology continues to advance, I believe that there will be an increasing focus on using AI to assist with assessment in higher education. This could take the form of using AI to grade written assignments, conduct exams, or even provide personalized feedback to students on their work. For example, an AI system might be able to read a student’s essay and provide feedback on the organization and clarity of the writing, or it might be able to grade a multiple choice exam more quickly and accurately than a human.

[incidentally, I experimented with using a avatar to speak the words of OpenAI  ChatGPT-3 and this is what I came up with from https://ttsdemo.com/ ‘oddcast’. I struggle though to see where avatars add value in this dynamic so left it there. Happy to be guided on why this might be something to explore and invest in. Transcript as per response to question 2 above].

3. Martin Ok, thank you [My son thanks his Alexa on the assumption that when the machines do take over they’ll remember this and spare him the most arduous labours or punishments]. What about now though? Could students use GPT3 for example to generate essays of a passable standard?

Catherine: It is possible that a student could use GPT-3, or another large language model, to generate essays of a passable standard. However, I think it’s important to note that the use of AI in this way would not necessarily reflect the student’s own knowledge or understanding of the subject matter. Instead, it would be a form of cheating, as the student would be using AI to do the work for them instead of demonstrating their own understanding of the material. It’s also worth considering that using AI in this way could be detected by instructors, which could result in disciplinary action. Ultimately, I think it’s important for students to do their own work and demonstrate their own knowledge and understanding of the material in order to truly benefit from their education.

Note the use of the repeated phrase ‘I think it’s important…’. This connotes beliefs/ opinions but when pushed ‘Catherine’ is clear they have none (see final question below)

4. Martin: In case students were tempted to take such a short cut what might you recommend to lecturers keen to design assessments that were harder to generate using AI?

 Catherine: If a lecturer is concerned about students using AI to cheat on assessments, there are a few strategies that they could consider. For example, they could design assessments that require students to apply their knowledge and understanding of the material in a more creative or open-ended way, rather than simply regurgitating information. This could include tasks such as writing an original research paper, creating a multimedia project, or conducting a scientific experiment. Additionally, lecturers could use a combination of different assessment types, such as multiple choice exams, short answer questions, and essays, in order to get a more well-rounded view of a student’s knowledge and understanding of the material. Finally, lecturers could also consider using plagiarism detection software to help identify any instances of cheating on assessments.

5. Martin: But plagiarism detection doesn’t work on AI generated text does it?

Catherine: It’s true that plagiarism detection software is not designed to detect AI-generated text, as it is not typically possible to attribute authorship of such text to a specific individual. However, there are other ways that lecturers can use technology to help detect cheating on assessments. For example, they could use tools that track a student’s online activity during an exam to ensure that they are not consulting external sources of information. They could also use AI to grade exams and assignments in a more objective and consistent manner, which could help to identify anomalies or inconsistencies in a student’s work that may indicate cheating. Ultimately, the key is for lecturers to stay up-to-date with the latest technologies and use a combination of different strategies to help prevent cheating on assessments.

6. Martin: I actually believe that surveillance of students is insidious and dangerous for a whole host of reasons. Do you not share that view?

Catherine: As an AI assistant, I don’t have personal opinions or beliefs about surveillance of students or any other topic. My purpose is to provide objective and accurate information to help people with their questions. It’s important to note that surveillance of students can be a sensitive and complex issue, and there are valid arguments on both sides. Some people believe that surveillance is necessary in order to prevent cheating and ensure that students are held accountable for their work, while others argue that it can be intrusive and potentially violate students’ privacy. Ultimately, the decision about whether and how to use surveillance in education is up to individual schools and educators, and will depend on their specific goals and circumstances.

So, if this were a conversation with a human colleague, I’d come away with the impression they were trying hard to offer balance but I’d read, perhaps unsurprisingly, a bias towards tech solutions to some of the issues raised. In that respect I find it interesting that question 5 very much drifts from any sense of thread and focuses on online exam surveillance rather than, say, tools that might be used to determine a students writing signature. Anyway, it’s a start. And very much the start of something pretty big.

*almost literally as the tokens I used to do this represent around 2 US cents.

Big tech headlights

Date
Posted by
Comments0

Listen (7 mins) or read (5 mins)

Whether it’s non-existent problems, unscalable solutions or a lack of imagination, we need to be careful about what educational technology appears to promise.

I have written before about how easy it is to get dazzled by shiny tech things and, most dangerously, thinking that those shiny things will herald an educational sea change. More often than not they don’t. Or if they do, it’s nowhere near the pace often predicted.  It is remarkable to look back at the promises interactive whiteboards (IWBs) held for example. I think I still have a broken Promethean whiteboard pen in a drawer somewhere. I was sceptical from the off that one of the biggest selling points seemed to be something like: “You can get students up to move things around”. I like tech but as someone teaching 25+ hours per week (how the heck did I do that?) I could immediately see a lot of unnecessary faff. Most in my experience in schools and colleges suggest they are, at best, glorified projectors rarely fulfilling promise. Research I have seen on impact tends to be muted at best and studies in HE like this one (Benoit, 2022) suggest potential detrimental impacts. IWBs for me are emblematic of much of what I feel is often wrong with the way ed tech is purchased and used. Big companies selling big ideas to people in educational institutions with purchasing power and problems to solve but, crucially, at least one step removed from the teaching coal face. Nevertheless, because of my role at the time (‘ILT programme coordinator’, thank you very much) I did my damnedest to get colleagues using IWBs interactively and at all (I was going to say ‘effectively’) other than as a screen until I realised that it was a pointless endeavour. For most colleagues the IWB was a solution to a problem that didn’t exist. close up of oldsmobile headlights in monochrome

A problem that is better articulated is about the extent of engagement of students coupled with tendencies towards uni-directional teaching and passivity in large classes.  One solution is ‘Clickers’.  These have been kicking around since the 1960s in fact and foreshadowed modern student / audience response systems like Mentimeter, still sometimes referred to as clickers (probably by older generation types like me). Research was able to show improvements in engagement, enjoyment, academic improvement and useful intelligence for lecturing staff (see Kay and LeSage, 2009; Keough, 2012; Hedgcock and Rouwenhort, 2014) but the big problem was scalability. Enthusiasts could secure the necessary hardware, trial use with small groups of students and report positively on impact. I remember the gorgeous aluminium cases our media team held containing maybe 30 devices each. I also recall the form filling, the traipse to the other campus, the device registering and the laborious question authoring processes. My enthusiasm quickly waned and the shiny cases gathered dust on media room shelves. I expect there are plenty still doing so and many more with gadgets and gizmos that looked so cool and full of potential but quickly became redundant. BYOD (Bring your own device) and cloud-based alternatives changed all that of course. The key is not whether enthusiasts can get the right kit but whether very busy teachers can get it and the results versus effort balance sheet firmly favours the former. There are of course issues (socio-economic, data, confidentiality, and security to name a few!) with cloud-base BYOD solutions but the tech is never going to be of the overnight obsolete variety. This is why I am very nervous about big ticket kit purchases such as VR headsets or smart glasses and very sceptical about the claims made about the extent to which education in the near future will be virtual. Second Life’s second life might be a multi-million pound white elephant.

Finally, one of the big buzzes in the kinds of bubbles I live in on Twitter is about the ‘threat’ of AI. On the one hand you have the ‘kid in the sweetshop’ excitement of developers marvelling at AI text authoring and video making and on the other doom-mongering teachers frothing about what these (massively inflated, currently) affordances offer our cheating, conniving, untrustworthy youth. The argument goes that problems of plagiarism, collusion and supervillain levels of academic dishonesty will be exacerbated massively. The ed tech solution: More surveillance! More checking! Plagiarism detection! Remote proctoring! I just think we need to say ‘whoa!’ before committing ourselves to anything and see whether we might imagine things a little differently. Firstly, do existing systems (putting aside major ethical concerns) for, say, plagiarism detection, actually do what we imagine them to do? They can pick up poor academic practice but can they detect ‘intelligent’ reworking?   The problem is: How will we know what someone has written themselves otherwise? But where is our global perspective on this? Where is our 21st century eye? Where is acknowledgement of existing tools used routinely by many? There are many ways to ‘stand on the shoulders of giants’ and different educational traditions value different ways to represent this. Remixes, mashups and sampling are a fundamental part of popular culture and the 20s zeitgeist. Could we not better embrace that reality and way of being? Spellcheckers and grammar checkers do a lot of the work that would have meant lower marks in the past but we use them now unthinkingly. Is it such a leap to imagine positive and open employment of new tools such as AI?  Solutions to collusion in online exams offer more options it seems: 1. Scrap online exams and get them all back in huge halls or 2. [insert Mr Burns’ gif] employ remote proctoring. The issues centre on students’ abilities to 1. Look things up to make sure they have the correct answer and 2. Work together to ensure they have a correct answer. I find it really hard not see that as a good thing and an essential skill. I want people to have the right answer. If it is essential to find what any individual student knows, our starting point needs to be re-thinking the way we assess NOT looking for ed tech solutions so that we can carry on regardless. While we’re thinking about that we may also want to re-appraise the role new tech does and will likely play in the ways that we access and share information and do what we can to weave it in positively rather than go all King Canute.

Benoit, A. (2022) Investigating the Impact of Interactive Whiteboards in Higher Education. A Case Study. Journal of Learning Spaces

Hedgcock, W. and Rouwenhorst, R. (2014) ‘Clicking their way to success: using student response systems as a tool for feedback.’ Journal for Advancement of Marketing Education,

Kay, R. and LeSage, A. (2009) ‘Examining the benefits and challenges of using audience response systems: A review of the literature.’ Computers & Education

Keough, S. (2012) ‘Clickers in the Classroom: A Review and a Replication.’ Journal of Management Education