Teaching accounting — the impact of generative AI

Toby York

So. Yes. Welcome, everybody. We’ve got a good number of people here today, about 60 of you so far, which shows that it’s actually a very important discussion to a lot of you. It’s very much on our minds, isn’t it? 

And we’re very lucky to have Stuart with us because he knows a lot about it. In fact, he never comes out of his shed. He’s constantly playing with these tools and finding new things. And I know he’s going to impress us. 

Just before I hand over to Stuart, what I just would encourage you to do, is just spend a minute or two to follow us on LinkedIn. 

But thanks ever so much again for coming along and making the time,  to spend with us here today. Stuart, over to you. 

Stuart Pedley-Smith

Thank you, Toby, and look, thank you all for . . . I’m kind of quite new to the community, so me and Toby have been chatting about different things, found some mutual interests Just a little bit about me, because I think it’s relevant in terms of the presentation. 

So as Toby said, the presentation says, my name is Stuart Pedley-Smith. I’ve worked in professional education for over 30 years. I worked for Kaplan for 30 years. So, and left around last September. Sort of my key role in those latter . . . when I say latter years, last 14 years I was their Head of Learning. So my interests sort of shifted almost from, if you like, the academic side of teaching in terms of the subject matter. And I’ve been sort of spending quite a bit of time looking at teaching more effectively.

That has bumped very conveniently into generative AI — ChatGPT. And I’ve been looking at that initially within Kaplan for about two years, but then since I’ve left, I’ve had a little bit more time. So I’ve really started to sort of try and think about its application within the world of accounting and finance. 

So my purpose today is not to be the person that knows everything, despite sitting in my shed and spending a lot of time looking at things. But to really give you some insight and inspire you to go away and look at some of these techniques. I personally think it’s revolutionary in terms of teaching and learning. 

And there’s two aspects to this. There’s one, I think it’s revolutionary in terms of teaching and learning. I actually think it’s a revolution in terms of the profession and how the job role will evolve. 

So those are my two big themes in terms of when you’re looking at the stuff I’m going to go through and as Toby said I want to do some demos. I’ve got some little introductions to do in terms of some basics, because I’m not sure what everybody knows. And then I want to just showcase a few things that I’ve put together for this presentation. 

So think about this. How can I use this to improve how I teach and deliver? So there’s a pedagogical side to it. But then equally, my other interest is, what does this mean for the future of the skill set of the profession? Because I seriously think it needs to evolve and reasonably quickly. Largely because a lot of the skill sets, in terms of analysis, can now be done by the technologies. 

We have been here before to a certain extent, but I think it’s going to enable people to do things without very much technical skill. 

In terms of structure, I’m just going to talk about the main GPTs. So I’m going to share a deck. 

Just very briefly I’m going to talk about prompting, which is the way that you communicate with the GPT. And then I’m going to move into the uses and examples and there’s only four slides. And then there’s a slide that looks at governance and culture. Because I can see these being barriers to adoption, more so than actually technical skills and abilities. 

So this is what I’m doing now. I work for myself largely, for a company called Synapse learning Solutions. And this is what we’re going to look at. 

So here they are. Now, let me just talk a little bit about what this is, what this slide is trying to show. Once again not knowing what you know is the tricky bit. But effectively you’ve got these things called generative pre-trained transformers. They burst onto the scene just over two years ago. Probably there’s two things to say about that heading. The key word in the generative pre-trained transformer is the word generative. So the way I always look at this is to say that if Google or Search, to be more specific, is a library, and the librarian goes and gets your books for you. So that’s the principle behind search. Generative pre-trained or the generative part, is somebody who has read all the books and just sits at the front and you ask them questions. 

So these models are not searching in the same way that a search engine searches, they are looking at their massive data sets and they are generating unique answers every single time. So that’s the kind of broad difference between a search and a pre-trained transformer. 

The other words that you can see up there are LLMs. And LLMs refers to large language models. So these are as opposed to when if you’ve seen some clever video content that’s been produced. Or you’ve seen all the furore over The Brutalist the movie, there’s also another whole new world of these sort of technologies taking place in the sort of  like the visual world. And of course, if you’ve seen Elton John, and like, you know, generating music. So we’re talking about language models, okay. Because largely that’s the tools that we use. 

What you can see is like — and I’m going to generally call them GPTs. So although they argue that they’re not, but they all are generative and they’re all large language models, I’ve picked four because these are the four that I’ve used. Everybody will probably have heard of ChatGPT. That’s the one that’s got the brand, and it’s actually got the GPT logo in it. Copilot, which is Microsoft, Gemini Google and Claude Anthropic. And as you can see, they all do slightly different things. But they will all appear at your first instance to do just the same thing. So they will appear to you, apart from the user interface to you, to be exactly the same, but they do have different skill sets. 

Equally, there are paid for versions. Everything I use is free because I think that’s how people come to it. So everything I’m going to demo today is actually free. But if you spend more money, you get deeper and better analysis. So there’s a lot of stuff around deep research that if you are heavily into the research, improves the quality. So don’t look at these and think it wasn’t very good. You know, it’s a bit like getting in a car that doesn’t go very fast. And saying all cars aren’t very good. You just got in the wrong car. 

The ones in purple, slightly different. They’re all, well, the first two in particular, I mentioned them, Perplexity, largely because it’s very good at summarising, very good at identifying sources. 

And Notebook LM is lovely. It’s a lovely tool. And what that does, this is the one that if you’ve got reports and documents that you don’t want to share, because there are a few warnings with this, and you do need to check with your IT teams and your AI policies, with terms of what you put into the GPT with something like Notebook LM, it’s a stored area so you can upload documents. So if you were doing research you can upload say 20, 30 research papers. And then the generative AI is trained — it’s only looking at them. So you can then just interrogate. So I’ve been using that quite a bit really on things looking more broadly at the accounting profession, putting all the sort of professional bodies’ documents in there and just sort of checking and, and just asking questions of it. And it will interrogate, it’ll pull out the information for you. 

Google AI Studio is there. A little bit more sophisticated, maybe, and a little bit more techie. But there is one really good feature in Google AI Studio, and it’s a screen share feature, because you’ll have to do a lot of teaching yourself with this because you won’t know what you don’t know. And you can share your screen and you can just talk to it as I’m talking to you now and you could say, what does this function do? How does this work? What can I do? It can’t move your cursor like a tech support could, but it will tell you what to do. So it’s a really nice feature that sits within Google AI Studio. 

So I suppose you know, my top tips are the ones that sit above, and then Notebook LM, probably and Perplexity for research if you want to look more closely at things like that. 

There are two little grey boxes on there. You’ll hear something called custom GPTs, and the equivalent of a custom GPT in Gemini is called a Gem for little gems. And custom GPTs are where people — think of these as apps. So because these GPTs are open source people have then used them to build things. So, for example, if you go into custom GPT store (it’s not called that, but it’s effectively like that), you can get a GPT that does perhaps anything. So for example, there’s one that’s . . . it’s a shopping one. Or there’s a restaurant one. There’s a . . . You talk to it. I love my mum. I love my dog and Aston Villa. Then it will design a tattoo for you. 

So the first part GPTs, second part: Prompting is the way that you talk to the GPT or LLM. Now, once again, I think most of these are self-explanatory from the text. The more detailed your prompt, the better the response. Once again, think of it as a conversation. If you say to the GPT, can you analyse these documents? It will analyse them and to a reasonable amount of detail. If you say, can you analyse these documents, but specifically focus on the sustainability issues, it will be better. If you tell it who it is. So you can see the one I’ve got there. So if you give it context or a character, it will behave as that character. So in this one, act as a CIMA tutor, act as somebody with, you know, many years of experience. Pretend that you are a student. Behave as if you’re a five year old. So context gives it the personality. 

And then you press the button. And when you press the button, it will give you an answer. If you don’t like the answer, you just carry on, say, oh, sorry, I didn’t mean five year old, I meant 15 year old. Sorry, I didn’t mean CIMA tutor, I meant ACCA, you know, I mean, somebody, professional education. 

And the last point there is about the format, really, or constraints. It doesn’t know how much content you want. So tell it. Can you do it in 500 words? You can ask it to do it in rhyme. You can ask it to do it in Chinese. Okay. So please summarise in bullet points. Some of them are instinctively designed like that. 

And you can actually customise your GPTs. So I’ve got mine slightly customised. So it responds in a certain way. So mine says a little bit about me: professional education. Can you please focus on learning science? So I always want evidence to support your statements. So mine is a little bit more personalised. But if you just going into it cold free model, then the more detail you get in terms of what format — can you put it in a table? Can you do it suitable to be exported into PowerPoint slides? It will do all that. 

And then just a brief example there, you can see, you know instead of saying — my  technical subject’s financial management, so apologies — explain capital structure in simple terms. So not complex, explain capital structures as it is tested by the ACCA using an analogy and provide an example relevant to an assignment. 

So you’ll get a much better response to a prompt that’s written like that. Firstly, the GPTs, secondly the prompts. Thirdly, the use cases. 

And look, you can use these for anything. They can produce content. They can mark. If you follow me on LinkedIn, I posted something only yesterday about RM, who are very confident that they’ve managed to get these GPTs to mark. They will be spending a lot of time and money in getting those prompts right. And they’ll be using a variety of different GPTs to do it. 

But you could certainly start looking at answers and playing around and just judging it for yourself. Once again, if you put the rubric in and you put a model answer in, and I’ve done this, you will get a better quality mark than you know. And I’ve only taken it to a certain point because it’s the kind of things you’re interested in. So it’s like how much time you willing to invest in that problem. 

And I think that’s the key point that’s at the top. Start with the task, not the tech. So okay initially take a photograph. This are these are mobile apps as well. You can get ChatGPT on your phone. Take a photograph of what’s in your fridge. It’s got order visual qualities. It’ll take the picture and you say can you do me a recipe that I can use with those ingredients? 

That’s just fun. That’s playing with the tech. But try to think of the things that you do. And if marking’s a chore or lesson plans are a chore or setting quizzes or all of those things, then focus on that and then use the tech, because it is very easy to get caught up in, all sorts of different things here. 

So you can do almost anything with it. As I say, lesson plans, support schedules, real world examples. Can you give me and can you give me the sources? I’ve set up a couple of chats. I’ll show you the one, talk a little what I’ve built around talking with Modigliani and Miller. Sorry. Capital structures again. 

But you can have ethical chat, so you could build one of these very, very simply where students could actually have a conversation, with somebody about ethics. So there’s lots of tools that you could use. 

I will just finish this in terms of a set section. Then I’m going to jump into some examples. 

Look, there’s a lot of other things to think about: bias, data protection, your institutional policies, you know, hallucination, all the things you would have heard. But, and this is how I tend to think about it, think of this as a very clever undergraduate or research assistant. They are very smart, but they can make a mistake. They don’t make many. But you would never do what they call exclude the human in the loop. 

Okay, so the human in the loop is everything. So you always stay within it until the day comes that you get confident. But that’s the same with any employee or any person that you’re working closely with. If they say it’s right, then it’s right. Okay. So these are when you see what they can do, they will make mistakes. So you’ve always got to be the final person that checks. 

And look, policies may be set by your universities or your institutions. And there is a big question mark around critical thinking, and you need to fiddle, sort of, look at those things and sort of get your thoughts together in your institutions as to how you deal with that? My personal view is people should be a little bit more less worried. 

It has issues for assessment, of course, but maybe we just need to look at assessment, a little bit more deeply. 

So those are the four things or the three main things really. What are GPTs? What is ChatGPT? They’re large language models. Secondly, prompting is the language that you use to talk to them. And this sort of issue around what you can do with it. It can do lots of other things. 

It’s a real leveler, I think, in terms of . . . so the way they look at it, they say it’ll improve an expert from 90% to 95, but it will improve an idiot from 5 to 60. So you could look like, if you took a subject that you knew nothing about, you could look like an expert relatively quickly to a non-expert But as an expert, it will improve you, but arguably, not that much, but that is always harder to improve towards the end, I guess. 

Okay. So this is the famous ChatGPT. Just register for it. This is the free version. If you look down the left-hand side here, you will see this is the normal GPT. 

Notice it’s now got search. So in the same way that Google is search and it’s gone into GPTs. We’ve also got OpenAI going into search. So there’s a real battle going on here between them, so you can search with this just as you can with Google. It’s not going to be quite as good. And they are doing those two different sort of exercises. One’s the librarian, the other one know s all the answers. So they’re two different things. 

You can see I’ve got some other things here. This explore your GPTs. This is these. Here they are. Look. Image generators. So there’s loads of writing. These are the customised GPTs. Scholar. These have all had the prompts written for them. Okay. Lifestyle, Song Maker, Python. So, you know, my coding skills are pretty much zero, but GPTs will write code for you. And they will check code. So that sits within exploring your GPTs. 

I’m just going to stick with the bog standard one. I’ll come back to these couple in a second. So I’ve got some questions set up here, just so you can see what you’ve got. Now, I’ll probably use ChatGPT for this. I will run out of credits on ChatGPT because you only have so many. 

So this is a question. And it’s just a pretty straightforward, sort of accounting question. So what you can do then is obviously copy that. So I’m just going to pop that in. So that’s the question. You can put sort of huge amounts of content into that. So that’s all I’m asking of this. I’m just saying, can you answer this question? 

You don’t need to worry about formatting or anything like that. And then say, if you’ve not seen this before, I think it’s kind of quite impressive. 

So my personal GPTs, most of them will do this. If you don’t get a step-by-step calculation, then you can ask it to do that. 

And as you can see there, the correct answer is 5.4. That was the answer on my piece of paper. And then you can just lift this.

And then just going back, just so as I remain consistent to my little tests, my next little test is could you produce a similar question relating to dividend policy but with different figures? And here it will go something like “Sure”. 

Look, it’s not the same. So look and see how it actually prompts you. So what’s the answer? I go, yes, please. 

There has been some research, although I’ve seen very different things — this is a real moving feast. I’m following this pretty much every day — that if you’re polite, you get a better response. Okay. And then I read something the other day, it said it doesn’t really matter. But what they’re finding is that,they don’t know why. 

I mean, we are a little bit in black box territory here, so, we don’t know all the things. It’s just that people have asked questions and then they get a better response if they say please. 

Yeah, you get a better response if you say things like, and this is what they refer to as prompt engineering. You get a better response if you say, this is really important to me. If you are not sure of the answer, please say so. So you get that sort of a thing as well. 

So this is ChatGPT. It’s, I think it’s pretty impressive. I’ll show you some of the things in a second. It can knock out questions and answers easily. You would check them, but you can cut these.  

In terms of copyright, big area for conversation. The current thinking broadly — and I’m not a lawyer, so please take this with a pinch of salt — but what I’m reading is there can be no copyright on anything to do with a generative pre-trained transformer. Where people get upset, is what it’s skimming. So if you say, write me a song like Paul McCartney, then obviously Paul McCartney might get a bit upset because it’s actually skimming off the music in the same way that we’re using language. 

So, you can’t claim this is yours because you don’t own the copyright. And therefore there are things like all of these answers, and you need to figure these things out. My personal view is you should just say, these questions and answers have been, sort of, overseen by me, but generated by a GPT. 

You’ve got to think of student values. You’ve got to think about what am I paying for? You’ve got all those things that’s . . .  I’m kind of politely sidestepping that. I’m just showing you the tech, you know, and what’s possible. 

Just while I’m on this before we move on to the next little exercise, I’ll just show you this. This is a Tutor Me one. So this is a . . . I just put this in my little group here because, I like this. This is a customised. This is Khan. If you’ve heard of Khan Academy. So it will just write you questions. It will just do. So this is actually designed. It’s got the prompts within it. It will help you. 

Can you explain net present value, see what it does with that? So in that sense, it’s got a tutor built into it. So, it’s kind of like, answering the question, but because it’s got a Tutor Me in it. So it’s giving you some examples. So I haven’t asked it to come up with anything. I’ve just asked it, does this make sense? 

Not really. Like let’s see what it does. So it’s,  so that and Khanamigo or Khanmigo as it is. And look what it’s doing now. No worries. Let me explain this to you. Let me break it down. 

Are you worried yet? 

Does that help? A little. So you can just talk to these things? I don’t know what it’ll do. That’s okay. So it’s just coming up with another one. So this is built within a  Tutor Me GPT so this is a customised GPT. Khanmigo, part of the Khan Academy, they’ve got some amazing stuff. One of the big places that this technology was launched, was firstly they went to Sal Khan and they said, can we start putting in that. 

Interestingly, to the best of my knowledge, they didn’t go to any of the big education institutions to try all this stuff. And, I think that was a little bit of a shame really. 

Would you like to try a practice problem together? So you can see what they’ve done with this is they’ve built it in such a way that it’s actually giving you answers, but it’s trying to teach you something. Yeah. 

The other one is Teaching Assistant. So this one will be able to . . . It sort of creates a blueprint. It will reusable prompts. So this will help you write prompts. Okay. So, if you say can you write a prompt for me, help me use a really usable prompt for an AI teaching assistant. So you can say, let me know which one. And you might say creating quizzes. 

So can you see how user friendly this beauty is, really? So, I said, can you write some quizzes? Now, I’ve not been very specific about this. You know, so what this is doing now, it’s trying to refine the prompt. What grade? You know, adult learners. Let’s try that. And as you build your way through it, sort of because I haven’t, but because I haven’t designed I haven’t gone in with the prompt. What subject? Okay. 

Now, can you see here I’ve reached my limit. All right. So because I’m using the free version, I’ve now got to go and pay. So, which is fine because it’ll tell me I can come back later. And for my use, that’s fine. I think if I was to buy one, I would pay for this. 

So I just wanted to show you Notebook LM. This is what it looks like. This is the one. I would like it in list form, so I can see. So, you can see here this is I’ve got this already set up. Now this is just a demo one I’m using for today, but, I’ve got the case study, but I’ve got some other resources. So this is the one I would suggest to use for research documents. 

So I’m eliminating that. So this is a CIMA case study. It’s public information. But this is where I’m saying is, please check with your team. But my understanding is if you put a source in here, it is safe, okay? As safe as it is anywhere. All right. So what a lot of companies can do now is they can put all their internal documentation. Now, they will do it differently to this. Because, once again, can I stress, I’m definitely not an IT expert, but you build, a safe space like this, you put all your internal documents in, and then you just have a chat. 

So if you say when can I take holidays? How many holidays can I have? What’s the policy for the company on AI? Can I talk to a student . . . and you just put all your policy docs in one place and then you just have a chat bot fronting it. 

So companies are not just using these in terms of, you know, external facing. So a lot of big banks for example. So you think of all your academic papers that you can put in here. And then you just ask a question. 

You can see it’s already . . . can you summarise the case study format. So this is about 18 pages. And you can see thinking. So here it is. This is the summary it’s produced. It’s a quoted company. So it’s already bolding things up. It does lithium mining, hard rock mining. So it’s quite comprehensive because I haven’t actually said to it, how long I want it to be or what it wants to deal with, but that’s taken a nineteen page document. 

So look, and let me use one of their questions. So this is, how does it manage currency risk exposure. 

Look, it’s got significant risk because it sells products globally. In summary manages currency through its in-house Treasury department. Exchange rate fluctuations. Ya-di-ya. 

Now it can only read that from the content that it’s seen. It’s only looking at this particular doc. The more docs you open up the more it will do. Yeah, I’ve got other questions, but I can just use their standard prompts. 

Look, the other thing to notice, look at the sources it pulls up. So these numbers here look, are telling you where it’s getting the information from, which is stunning I would argue, absolutely stunning. 

And of course once you got that in you can put the whole of that in and say, can you write me a case study similar to this but based on the car industry? Put a load of car industry data in here and it’ll offer you a pretty impressive first draft. 

One word of warning with this, that will all disappear. So what you have to do is save to notes. And you can see in this here, it’s now saved that, because when I cancel this, this will all disappear, which is what gives it, partly gives it, its security. You can of course, just copy it over into a Word doc and take it away from there. 

So that is Notebook LM, different to a GPT model. Because it enables you to do this. Oh, I forgot actually, the funniest . . . the fun bit. It will actually generate you a podcast. And, they’ve started — this has changed within the last two months. You can actually intervene in the podcast, but if you press that. Okay, it’s a bit American. And having worked for an American company, I know exactly what that means, but it’s very, very good. It’ll go okay. Hey, hey, isn’t this a really interesting case study? I, you know, I think, Rotomyne is a really sustainable company. What do you think Louise, and Louise will say? Yeah. I’m particularly interested in this lithium petrol . . . And it will actually put quite a lot of detail into what’s actually going on there. So please play with that. It’s free. Okay. Everything I’m showing you is actually free. 

The last thing I’ll show you, is actually, something that I’ve just been playing with. You can see here, I’ve . . . have a chat with Warren Buffett, George Soros. So this, I’ve built this through ChatGPT Teaching Assistant. But I thought for me, I’d do a conversation with Modigliani and Miller. So this is the size of the prompt, look. So it’s taking the prompt so I can put that into any GPT. It’s not I suppose it’s not massive. And actually, let me use Perplexity, so you can see what that one looks like. 

So similar setup, Perplexity. So I’ll put that prompt in. So it sits there and then I’ll press and then let’s see what it does. Okay. 

Hi . . . So it’s now talking to me look.  I’m very happy to answer your question on Franco Modigliani. 

Why did you say that companies should be 99% geared? That’s rubbish. And that’s a bad idea. Let’s just be a bit polite. 

So you never said that companies should be 99. They would indeed be very risky and potentially . . . What I did argue with my colleague Franco Mondigliani, in an idealised company, the value is unaffected by the . . . Ah, I’ve misunderstood that. Sorry. 

Think of it like a glass . . .  look at the analogy. Normally, it does pizza, to be honest. Water . . . da-di-da-di-da.

And then it says, what are the potential risks of having and . . .  you’ve got loads of prompt questions. It is stunning. I think. Okay. 

Toby York

Thanks Stuart. That’s pretty amazing, actually. I’ve learned a lot. I thought I was sort of a little bit on top of this, but of course I don’t think anybody is really on top of it, are they? It’s massive. As somebody just said, it’s a little bit like the start of the internet. We’re all on this road of discovery. 

There are lots and lots of questions in the chat, and we can go through some of the specific ones. But I think, if I was to draw out a particular theme, one of the themes is ethics. Ethics in terms of putting student work into ChatGPT . . .

Stuart Pedley-Smith

Yes. 

Toby York

. . . for example. And the effects of using AI given the huge energy use and also how these language learning,  these — what are they called? 

Stuart Pedley-Smith

Large language models. 

Toby York

Large language models, see I know even less than I thought about this, how they’ve learned, you know, there’s some ethical questions about how they’ve actually developed. So all sorts of ethical aspects. I don’t know where you stand on that or what you think, whether it’s even possible to use AI ethically. 

Stuart Pedley-Smith

I think, I’m probably a little bit more relaxed about certain things, but I come from a slightly different sort of side of the fence in professional education and less, you know, more about external exams and things. 

I think . . . look to me it’s about policy. So the students will be using this and I saw a couple of questions, feeding case studies into . . . Nothing, So . . . actually nothing stops them. Which is why your AI policy doc needs to tell them what they can do. So when I write some of the Kaplan stuff, one of the things, initially and it was a draft, this was about 18 months ago, I just put when you submit your answer, will you tick a box? And it was, I’ve not used any generative AI for this. I have used generative AI to create an outline plan. I’ve used generative AI to create an answer but I’ve substantially reworked it and I’m happy it’s my own work. 

So there’s a way of morally laying that back on the student. Now, you know . . . And then at the same time, giving them the framework for which they can use it. So it’s okay to use it to do an answer. Yeah, it is, but then you’ve got to think about, well, what do you mean by substantially reworked? And if the idea is, yeah, are the ideas your own in the first place, if you spent two hours on the internet reading what everybody else said. 

So, so there are, there are questions to answer. Things like putting student scripts in is to me a little bit about policy. So you can just remove names okay. So you know, you could just, you know, anonymise the content so that there is nothing possibly stored. But and once again, I think it’s a little bit about your own internal data security. What do you do with scripts now. Do you put them on an online platform? You know, so yeah. You think that professional bodies, they’ll just give them numbers and they’ll upload and so you wouldn’t know who they were. 

So I think some of that is old news. We’ve always had to deal with that part of it. Some of it in terms of . . . well there’s two sides to this, isn’t there? One is, what can students do? And the second side is, so what are you going to use it for? 

And I think in terms of that last slide. Yeah, your governance, your policy document just needs to be very clear because students will be confused. Do you know what I mean? They’re not going to know if you don’t tell them exactly. And if you say don’t use it, they’re going to ignore you. So you’ve got to review assessments. Assessments have got to be reconsidered completely and there’s big moves to sort of look at that. 

And you can use some of the Gen AI to do it, so you could explore a conversation they had with Modigliani and Miller, and mark that. 

Toby York

The other . . . I’m just going to put a couple of other questions to Stuart. And one is about, well actually they’re two parts of the same question, which is about how do we learn more about this and how do we help our students use it to learn? 

But if you’ve got other questions which I haven’t synthesised well from the chat, then please do put your hand up and I’ll come to you and you can come and ask Stuart directly. 

So there were a couple of comments in the chat about learning. Specifically, how do we become better at prompting? Are there courses? Is there a program we can put ourselves through to help us do that? And how do we support our students to use it appropriately and effectively? 

Stuart Pedley-Smith

Yeah, two good questions. I think. The first one, look, this is where I should say fortunately, Toby, thank you for inviting me today, I’ve got a lovely course for $500. I haven’t actually, but, there’s lots out there. They are all offering . . . There’s a load of people jumping on the bandwagon. I haven’t actually done a course. But I have had more time than perhaps some people have had. 

So when my wife asked me, what have you been doing? I say I’m “upskilling”. I’ve been upskilling now for months, I say, I’m still upskilling. 

So, and, you know, I use any available source I have been using, and YouTube is great. So this morning I was looking at. Yeah. So things I do, I look at YouTube and I look at the date. So if it’s three months out, that’s probably my limit. Right. And they’ll be . . . if you go on YouTube, there is just tons of people giving explainer videos and they’re bite size and they’re generally good. They’re doing what I’ve just done basically. And that’s pretty much how I’ve learned. I keep up to date with a series of things, probably a little bit more geeky than the average person would be. 

But in terms of this is an institutional question, Toby, I think because, you know, I think the institution, the company that you work for, has got to take some role in actually saying, look, this is the view. I don’t want to stifle anybody’s imagination, but everybody’s going to want to know this. 

And as I said at the start, I think the slightly broader question is, is what we’re doing with it is but what you should be teaching them to do with it? So when you’re setting a question you say yeah please use ChatGPT, write the prompt. Isn’t that what we should be teaching? You know. So it’s giving them the answer. 

So then you’ve got to come up with this alternative method of assessment that validates whether they’ve actually understood it. You know. So a lot of people are, you know, you can do viva type presentations. So there’s lots of different ways of looking at it. Also you can have conversations and check the conversations. You could look at forums and see how things have been written. 

But there’s very little to stop somebody producing content that looks like they’ve done it. And the other thing, nobody, Turnitin or nobody I’ve read, and I follow a guy called Ethan Mollick, and he’s like the god of all this. And he says there is no evidence that any checking software can spot. Because you just put in the prompt, can you write this in a way that won’t be picked up by any software, and it sort of produces it. 

Toby York

Yeah, yeah. 

A couple of other interesting points. Please, put your hand up and come and talk. I don’t have to be your filter because I’m going to do it badly. 

But yeah, David Henderson saying quite interesting that in the early days of calculators they were banned then tolerated, now mandatory. So maybe AI is going to take a similar journey. 

But in the context of the profession itself, what do you think this is a question from Wendy. What do you think we should be telling our students or teaching our students in terms of the future of the profession? What are the professional bodies up to in this space? 

Stuart Pedley-Smith

This is from my experiences. They are . . .  it’s difficult for a professional body. They’re okay in assessment because they’ve got, you know, okay, we’ve got the remote proctoring. But that’s tied down relatively. And then they’re taking people into a locked room with a computer and they’re saying, do my test. So in terms of assessment, they don’t actually have to do anything in terms of actually assuring that it’s the individual’s own work. 

In terms of skills, every syllabus has got something on AI. What it hasn’t got is, can you use it? So there’s a very sort of essay approach that I would say is, yeah. Can you explain the principles of AI and how it might be used to produce a set of accounts? 

It’s quite difficult for them without massively changing. And look, ICAEW moved towards this in terms of the audit software. If it were me, I would start looking at building into those online assessments something that says here’s the GPT, you can use it. 

And that then causes a problem because then they’re not sure it’s their own work. So maybe you could lock elements of it down. So in terms of an external exam. 

But to answer your question, I think, if you move into a professional office or into any finance environment, this tool is like not being able to use a spreadsheet. Yeah. And especially as the employer, it’s going to take, you know, your part of your job could be speeded up considerably. 

And so to me it’s all about getting ahead of the technology. If not, the technology will get ahead of you. 

So in terms of curriculum, you know, and this is why, you know, when we started speaking about this, Toby, this is why I think this is such an important forum. And I don’t see any big voice just within our sector. 

The professional bodies are putting out information. They’re covering off as much as they can, I would argue, in terms of what is it, how might you use it, how might it change the workplace? But they are very much thought leadership arguments. 

The practicalities of when you get in the office on Monday, should we be putting clients accounts into this? Because you can, I mean, I’ve done this so you can put a full set of accounts and you can just say, can you . . . like with that case study, can you identify the key ratios, produce me a report for the client that actually identifies areas concerned. And look I’ve already looked at it. So I might say I’m particularly worried about the levels of cash balance. The client’s called Richard, can you produce me a letter in our standard format? Bang. 

You know, you would kind of like, and somebody who’s newly qualified and I’m thinking of the professional education market now, to actually coming to your office to people our age. My age, I’ll say Toby, because you’re still young, and coming in and actually saying to me, why are we doing it like this? I’ve just been to college, I’ve been to university, and they’ve been showing me a far better way of summarising. Why are we still doing this? And working it in like that. 

But that’s why I say it’s a little bit of a revolution. 

Toby York

I think there is a broader question, also not necessarily about what’s happening in the accounting profession with regard to training, but what’s happening in the accounting profession generally, what’s going to happen to accounting? 

Stuart Pedley-Smith

That is a big question. 

I . . . this is one of my areas of interest. Now I’ve got a bit more time is actually getting ahead of the technology, as I’ve said, in order that, you know, oh, we won’t need accountants. 

Because you’ve got to go back to core competencies, haven’t you? Sort of like analytical thinking, critical thinking. These skills don’t go away. And accounting teaches attention to detail. Those are still very valuable skills. 

So I’m very optimistic about accounting because actually when you look at this technology, it can do everything. 

I was listening to a presentation in a podcast actually, two days ago, one of the guys in there, a guy called Nick Shackleton-Jones. He’s from the L&D space. He was saying that, you know, it’s not that the technology will take your job. It’s just that it’s classic going on holiday moment. 

So if you set it up and say, can you take all my emails, and respond to them, with looking at all my previous emails, and updating them with the last set of financial information, and you do this and you set it, and then you go on holiday and then some day you come back, they say, well, actually, we didn’t know you’d gone. 

You know, so it’s sort of like boiling the frog moment that actually as you automate more things. 

But you got to get . . . this is a rising water. You know, you’ve got to get above that water effectively and actually say, no, you know, I hope I’m not self-justifying the profession here, but I still think this is a tool. 

Because if you take it to its Armageddon logic, then actually it will do all these things. And the next phase of these GPTs is what they call the agents. So this is where you link it to your calendar, because you can write a prompt that will link to your calendar or other things. And you could say, I want to go on holiday to Africa, probably the first two weeks. Can you find me some flights? Not necessarily the cheapest, because I would like to travel Business Class. The shortest route namely for me, two others. Find the cheapest price. I think a four star, not a five star hotel. Please. And the agent will do it. 

And that’s the next step. In terms of where these pre-trained transforms. So this agentency is that called. And, yeah, this agent that will be able to do a series of tasks. So that’s what the techies are working on, now whilst we’re sort of getting to grips with the the technology as it stands. 

So I’m optimistic. I would say. Yeah. 

Sally Ahmed (Queens University Belfast)

So the first question, have you guys used any of these tools like during the class? And it was like very rewarding in terms of the students’ engagement? This is the first one. 

The second one, I can’t remember, I think you mentioned related to the marking, which tool have you used for marking? And was it very useful? 

Stuart Pedley-Smith

I’ll jump in Sally, on that. Thank you. And to answer your question, I’ve played with a few like the Modigliana & Miller thing but initially it was really just saying to students, if you get stuck, just don’t ask me, put it into ChatGPT. 

Now that does put you on the spot, actually, because it actually says well, ChatGPT says this. So the days of sort of knowing the material and saying, well, I know where all the mistakes and everything now you’re dealing with a live situation where you can say you’ve then got to look at the GPT, but they’re going to do it anyway.

And using it as a tool to validate or check, particularly if you looked at that coaching model that actually took them through it, would be a great tool for them to use outside the classroom anyway, so using it in the classroom is not a bad idea. 

So that’s the only thing that I’d used it for because it’s been about two years now. 

And the second question, I just use ChatGPT actually. So you write, I put the marking grid, so I just use publicly available stuff just to test it. So I’ll put the marking on. I might have done it in Notebook actually. Notebook LM, I think, I put the marking guide, a model answer, the question. And then what I did is, I went on to ChatGPT and I asked ChatGPT to write me an answer that the student would write that doesn’t score very well, and I gave it a 50% mark. 

I took that answer. I put that back into Notebook as an experiment. And then I said, can you mark this? And it did and it marked it well, but I’d need to . . . personally I’d need a student answer to validate the test. But it did it and I got it. So I got the prompts because I just kept improving the prompt. Can you produce the rubric in a box? Now, can you present the percentages at the top of the page, not at the bottom? Can you make it light hearted but clear? Can you make it motivational? I don’t want students . . . 

So all of the feedback, I started reiterating the prompt and I got to a level with it. But in terms of, have I done it for real? No.

But I think that’s the point where you’ve got your marked answer and, you know, you use that marked answer, you put that one through and you then compare. 

Toby York

Thanks ever so much, everybody, for coming today. It’s been wonderful hearing your own experiences. As I said, it would be really helpful if you could, follow Accounting Cafe on LinkedIn. Follow that link. Or just search for Accounting Cafe on LinkedIn. 

But, as far as the formal session is concerned, then that is it. So thank you all very much for coming.