Introduction and welcome

Host (Toby): Welcome, everybody. It’s a real delight to see you all, and thank you very much for taking the time out of your busy days to come along and say hello and join in with this conversation.

For those of you who aren’t aware of Accounting Café, I’m going to take less than a minute to talk to you about us. We’re a community-based organisation. We don’t have any sponsorship, there’s no membership fees, there’s no advertising. It’s a labour of love at the moment, and we’re thinking about how we make it sustainable for the long term.

One thing that you can do, which is enormously helpful for our reach and our presence, is to follow us on LinkedIn. So if you don’t mind, if you don’t already follow us, just take a second to scan the QR code and click on follow LinkedIn. That’s just a free and easy way to support the work that we’re doing.

Today, I’m really excited to hear about these EdTech strategic partnerships. Judith, are you going to talk a little bit about how we’re going to structure this session? Because we’re all going to be out of here in an hour, aren’t we?

Judith: Basically, I think we’ve got a few questions to run through, and we want to be finished up with about ten minutes or so for question time. We’re going to start with Danielle telling us all about her work. The reason Danielle is so interesting here is not only is she a very enthusiastic EdTech advocate, but she was also one of the contributors to the new benchmark statement. So she’s seeing this from a very interesting angle, which I think has something for us all to learn.

Toby: So, Danielle, tell us about your experiences and how you were using the system.

Prof. Danielle McConville

Danielle: Hi, everyone. Thank you so much for the opportunity to be part of this conversation today. I’ll talk slightly broadly about learning technologies first of all, because I have a huge interest in this. I’ve been teaching in the sector for a long time, and as you mentioned, I’m head of department within Queen’s Business School for accounting.

When I think about what I’ve been doing over a number of years in terms of the educational technology space, broadly speaking, that splits into two broad areas. The first is how do we teach technology to accountants? That’s largely what we’re going to talk about today. But actually, I’ve also been involved for a very long time in using technology to teach accounting to both accountants and to non-specialists as well. From that perspective, I’ve been involved with things like Pearson MyLab in the past and our own technologies that we’ve used like Canvas quizzes. We use a tool called Numbers to use algorithmic questions.

But actually, all of that comes around to the same point, which is around helping students to develop. I have a very big heart—I am a chartered accountant myself—for developing the students who are going to be the future of our profession, not just thinking about their first graduate job, but actually thinking about people who are potentially going to be the leaders of our accounting profession as we move forward and will safeguard our accounting profession as we move forward as well.

That influences a lot of what I do because I’m not just interested in making sure if I’m teaching accounting, they get the debits and credits, they get how to do variance analysis. I do a bit of management accounting, or indeed that they know how to use XLOOKUP on Excel or they know how to use certain things within Power BI. Actually, what I want them to do is I want them to think about accounting information as a system. I want them to think about where that information is coming from, what we’re doing to that, how we are conveying that information to people, and actually to be constantly thinking critically about stakeholders and all of those sorts of things that are really important.

Because, you know what? Whatever we teach, particularly in the technology space, I’m really aware that by the time they get out into the big bad world, what I have taught them will have moved on. So there’s a lot around teaching people to learn in the higher education space. How do they engage their self-efficacy, either with technology or with accounting? So how do they build their confidence and how do they learn to learn technologies? So there’s a lot of that kind of where I’m coming from.

QAA Subject Benchmark Statement on Accounting

I brought all of that into the Subject Benchmark Statement. If folks ever get an opportunity to be part of an endeavour like the Subject Benchmark Statement, if you’re an engaged and interested educator, I can heartily recommend it. I had so much fun. We genuinely had the most involved and interesting conversations back and forth. While I would very much be somebody who has a real heart for educational technologies, I’m also head of department. I also understand that actually we have resource constraints. We don’t always have the right people to be able to teach certain things. We also have budget constraints. We also have a whole range of things.

Actually, quite often we have people perhaps in our groups who are not fully bought into the need for either us to teach technology or for us to use technology in our teaching. Sometimes the criticism that gets thrown is, “Oh, well, this is all about teaching basics. This is about basics of debits and credits, or this is about the techniques around Excel and XLOOKUP.” And actually, what we have to constantly counter is the fact that it’s not about that. It’s about learning to learn. It’s about adaptability. It’s about professional skills and values and actually being able to go beyond those that the technology can help us to teach and inculcate those values, but also it can help us to go beyond and really to develop our students as our future professionals.

Within the accounting Subject Benchmark Statement, we were challenged with five what they call transversal themes. These were themes that we had to embed within all aspects of the subject benchmark. This comes from the QAA. So all Subject Benchmark Statements now have to deal with these five transversals. For example, finance is also just done its Subject Benchmark Statement and got handed the same five.

One of the five we got handed was AI. Now, for me in accounting, I felt like that’s actually kind of narrow because actually technology in accounting is so now pervasive as part of our profession that actually we need to think not just about AI, but about various other aspects as well. We need to think about systems. We need to think about data. We need to think about analytics really particularly. AI is actually kind of to me the end of a spectrum where actually there’s a whole lot of other stuff in there.

So we thought very broadly when we did the Subject Benchmark Statement about what it meant for students to be digitally fluent or digitally literate. It wasn’t necessarily about being prescriptive about technologies. Actually, instead it was about things like systems thinking. It was about thinking about being able to manage change, to be adaptable. We also put quite a lot in there around employability and lifelong learning as well, because that’s so critical to us as accountants. CPD is so critical to all of us, and actually particularly in the technology space. I know when I talk to people in the profession, a lot of the CPD that they’re doing at the moment is in the technologies space to try to keep up.

So what we want to do with our students is to make sure that they’re well set up to be able to learn that way, and we wanted to put something of that into the Subject Benchmark Statement as well.

I am aware that the Subject Benchmark Statement is hopefully—I mean, we perceived it as something that we hoped would be useful. We appreciate that there is going to be some challenge and some stretch for a lot of institutions. But I suppose it’s worth being aware that it really does come from the place of us wanting to think about the well-rounded graduates and future contributors to our profession that are going to come out of our higher education institutions.

So I hope that’s helpful around the Subject Benchmark Statement.

Toby: That’s really helpful. Judith’s going to talk a little bit about how the sort of partnerships work. So can you talk a little bit about that and how your relationship with them has developed?

Queens University Belfast — partnership experience

Danielle: Yeah, sure. I mean, I started working with Judith and XingDong originally for—I had a new Level 3 module where we wanted to have Excel and Power BI, and we were really struggling to know how we would teach and both deliver that and also assess that at scale.

Toby: Just explain to people what Power BI is? I think everybody knows what Excel is.

Danielle: Power BI is data analytics software. We originally started off with Tableau data analytics software, and we moved to Power BI probably about 2 or 3 years ago, mostly because it suited our institutional Microsoft licence in honesty. So our students already have access to Power BI through their Office 365 licence, as I suspect will many of the students of the people on this call as well.

So we wanted to have Excel and Power BI. This third year module’s actually at the top of the technologies pillar for us within accounting because we introduce—first year we have Excel and actually it’s Xero accounting software at the minute. Actually, that’s done through the AccountingPod as well. That was a little “Brucey Bonus.” We then have in second year we have optional modules on either Excel or Power BI. And then this was going to be a third year capstone module. We were running the numbers—we were expecting numbers of about 100.

The challenge that I had was that I had not—when I’d taught things like Xero and Sage before, I’d not taught Excel and Power BI, and I was also thinking about how to assess it, because my experience of Sage and Xero is a very manual marking process where literally all I was able to do was print off or show on screen more easily, final statements and mark the final statements. I couldn’t mark in any way what students were actually doing with the software. All I could mark was the final output, and even then it was very tricky to mark that at scale. So, again, about 100 in the year group.

Thinking about both of those things, I got chatting to Judith at—it was actually the CABS conference. What we did was, I started to work with them.

Broadly speaking, what we have is we have two companies. We have a practice company and an assessment company. The practice company is real data and it has a lot of guidance for the students. What I do is I work with the students in class. So I do four 2-hour lab-based sessions with the students in the class where we work through the content that’s provided by AccountingPod. I add some bits and pieces here and there. So for example, I go off and say, “Now you do remember what a regression analysis is,” and I’ll go and show them something. So we do bits and pieces like that, but we also troubleshoot in the class as well. “Oh that’s not worked for you, okay let’s look at that. A few of you are having the same problem here.” You know the way it goes.

So I work through the practice company with them. They also, as soon as they submit work, they get automatic feedback on that work as well. So it will tell them exactly where they’ve gone wrong and they can go back and fix that either in class time or in their own time. But we work on that over about four weeks.

Then they have an assessment company. So at the end of the classes, this assessment company lands again, it’s real world data. It’s a real world problem. They’re asked to work through that assessment company. It includes Excel and it includes Power BI.

With the Excel, they have an upload and the upload gets marked through AccountingPod. With Power BI, there’s actually an API link into their Power BI, the version that they’re working on, so that the marking is not just a final output. The marking is actually working through what they have actually done on either Excel or Power BI. So the marking is at a level of detail you couldn’t imagine doing. You physically actually couldn’t do it. So it’s doing something that actually you manually couldn’t do, but it’s also doing it at scale automatically.

What they were also able to offer me was they were able to give me algorithmic questions. So different students had different data sets. So it dealt with some of those issues around integrity. They were also able to build for me groups, which was something new—I appreciate it was a bit of work at the time. I was really keen to build in collaboration between the students who had maybe some of them had done a specific Excel module previously, and some have done a specific Power BI, and I wanted them to work together like they would in the real world to share that knowledge, albeit that they all had to do a version of this. So they were able to do that for me. And again, those were algorithmic. So each group had a different data set to another group.

So that’s how we have used particularly on the Level 3. I suppose the big benefit for me was for me coming into teaching something that I hadn’t previously taught before, albeit that I’d used the software and was reasonably familiar with the software. It’s scaffolded my teaching. So I was able to rather than start from a blank sheet of paper, I was able to run computer labs based off of the really good resources that are built into AccountingPod, which, by the way, draw in a whole range of different sources, which is really nice. So they link into it, not just all their content. Here’s how to do X. They’re videos from Microsoft, videos from different places. There are also some things that students have to go and find out. So just like us in the real world, “How do I do that? I’m going to go and Google. I’m going to go and find out. Oh, how do I do X and Y?” So there are little nuggets of things that they have to go and find, which again is really important in that space of learning to learn technologies.

So we had all of that in the teaching. And then also in the assessment, it’s an authentic assessment. Real world data set helps me with a lot of integrity issues. And also, does most of the marking for me, so it does all the technical marking for me.

I should say the thing that it doesn’t mark for me currently is that we asked students to prepare a report at the end where they advise the client. So again, going beyond, “Can you do the technical things?” And actually say, “Okay, can you give the expert client some advice about where they should buy these properties or what they should do in terms of X and Y?” And it also asks them to do an elevator pitch. So they have to record a one minute pitch where they briefly summarise their ideas in one minute and record it to AccountingPod. I currently mark all of those manually, albeit that I developed a little rubric to do that. But what I got told this morning is that my rubric has been incorporated into AccountingPod. So that will make my marking even easier next year.

Toby: That’s great Danielle. It’s very, very interesting—it’s very inspiring as well. Your enthusiasm for what you’re doing and for the teaching.

Some background on AccountingPod

Toby: Judith, it would be quite useful to hear from you, perhaps about how you go about co-designing with educators and what the AccountingPod approach is to building what Danielle is talking about.

Judith: Maybe Toby, a little background about why we thought, “Hey, surely when you have business tech stacks, surely there should be an educator within the business tech stack.” So that’s essentially what AccountingPod’s vision was—to close the gap between accounting theory and current business practice.

Now, I’m talking back to the days of our start up with the great big idea. And so how do we go about doing that? What we learned was there were hundreds of these small platforms that sat around cloud accounting platforms, doing different things for different sectors. They were connecting via API, often the pipe—and that means data can flow. Often that pipe is only one way. We wanted the pipe to be two way, because that would mean we could take a cloud accounting platform like Xero, or Sage or QuickBooks Online, and we could actually pump data into it and we could retrieve data at scale.

The concept that Danielle’s talking about is why we can plug into the likes of a Microsoft Power BI, and basically plug into the back of the students’ instance, and provide unique data, which is algorithmically varied. So they each get a different set. So we’re very big on this integrity approach.

I guess it’s been a few years working through that, Toby, so that today Danielle can talk about how her students can have that real time feedback and we can meet one of those key goals around authentic assessment. And also, I guess, Suresh here has talked about student psychological safety because he’s got a bit of background in that area as well. So we understood if we could provide the learner with real time feedback or as close as possible, then we could really help with that angst around these new products.

I mean, I think, Danielle, the Benchmark Statement is very clear around it’s appropriate to be using digital technologies for the students to really get hands on. And that’s what we used to talk about. But let’s get the students hands on and build case studies and stories of business, but get them doing things in these different platforms.

So we were very keen to be able to offer and be agnostic to these platforms. And why do we need to do that? I mean, we think it’s important because it helps students with agility as well across one platform to another and understand what it’s up to.

The other thing we’ll get on to AI shortly, but I remember the later part of COVID, everyone was talking about how all these platforms wanted to be the next Fintech. So, you know, you take, for example—I’m trying to think of a simple example. Say you’ve made a booking for your hairdresser and you did it online when that they actually now want you to just maybe pay upfront for that appointment. They quote, they set the appointment, and you may then have to pay a deposit or whatever it is. But I mean, what’s happening there is a payment gateway has been activated.

So we see a flow of data through what was a booking platform now adding on some fintech. Because let’s face it, where the cash flows, cash flow is king, I guess. I think that they are next generation of graduates are seeing and understanding what this technology is doing and where it’s headed. So that’s why we were very keen to see them actually using it and able to have these insightful conversations with the likes of Danielle, the lecturer.

So from our perspective, how do you grow an EdTech within accounting and business accounting, finance. And I guess this is education because we thought business financial literacy right across any university actually is just as crucial as financial literacy on the private side. And so there’s great opportunity there.

But how do you do this? You do this by talking to people, listening to the faculty and thinking about how to respond. And that’s basically what our team have done, and develop a team. We’re always iterating, we’re always releasing something. And that’s actually why Danielle was given a little present today in terms of, well AI came along, we thought, “How can we grade this or how can we auto-grade this written task and this audio task.”

I mean, I remember the early days of going about all the universities in New Zealand and meeting with the head of school and sitting down and meeting other lecturers and seeing these huge piles of paper. You reminded me, Danielle. It doesn’t make any sense to me. If technology can solve grading at data point and feedback at data point and we can vary those data points to ensure that everyone gets a different set, or we know that that person is the one using that cloud accounting ledger, because we track and key integrity features around it, and it just doesn’t make sense. We needed to think of how to release academic staff in the classroom away from grading. And we’re really pleased to have achieved that.

Yeah. So it was a big focus when we first started out.

The co-design process

Toby: What I would like to know is, Danielle was talking about a very specific way of using the platform that you’ve developed. So she was talking about how it fits within the pillars of their program and where it sits and what she’s using it for, and then which program and for what assessment. So how do you actually go about co-designing how this works with an educational institution? What’s the process? Because it’s not simply just giving somebody the keys and saying, “There you go, off you go.”

Judith: No, we often normally do have an early conversation and just talk about what’s being taught in the course, maybe what textbook is being used or what resources are being used and what they’re really looking for. And then, I mean, we’re not bespoke designing. We’re basically building library content and sometimes we are working alongside an academic and saying, “Well, actually, you’ve got an interesting piece of material. Could we bring it across onto our platform?” So we can therefore respond to the faculty need.

And also if you were wanting to think about how would you embed—who was talking about this? The data literacy mindset across your curriculum, you would want to be doing everything in each year, wouldn’t you? And think about the design system. So we’ll actually sit down and work out something that makes sense and then look at available case sets and that will also give us an indication of new case sets we need to be thinking about in different accounting areas. So the management accounting and the intro accounting and here data analytics and accounting information systems. And so it goes on.

Toby: Super. Thank you.

AI and authentic assessment

Toby: XingDong and Suresh if you’re available, you were going to talk specifically about how we maintain authenticity of assessment in an AI world and whether or not there’s a way of dealing with those challenges and problems that we’re having.

Suresh: I’m happy to kick off and so authentic assessments, right, Toby, that’s the key area and potentially there’s an area of—not area, mindset—of total uncertainty. Where is AI taking us? It’s almost a fear I’ve found in talking to some academics, and rightly so, because the public is worried about AI safety.

So let me take just a second to why I think AI is actually a saviour for us from an accounting education perspective right now. What we found is the role of accounting, of course, has become more and more automated, but we now live in a world of narrative and as educators…

Toby: Sorry, Suresh, we’re losing you. I’m going to come back to you. We need to find a place where the camera’s not so wobbly and the sound is not going in and out. What you’re saying is fantastic, but it’s too stop and start at the moment. XingDong, just pick up for that while we try and get Suresh sorted out.

XingDong: Yeah. So so when I talk about, I think authentic… also sorry, I just wanted to respond to a point. I think the question you asked earlier, Toby, about how do we go about this with the educators to deliver what they need? I remember one of the very early conversations with Danielle was that there was a clear kind of data points coming from the employers what they wanted the graduates to be able to do in the workplace. Danielle, I think, yeah. So I can see you nodding to that.

Danielle: Yeah.

XingDong: So based on those needs, we canvassed what we needed to do and whether it’s in our library of stuff or with the things that we need to be able to bring to the table to help and to essentially meet with what employers would like their graduates to come out of the degree to be able to do.

So I just wanted to make that point—the collaborations, the conversations.

So when we talk about authentic assessments, I think we have to talk about marking and grading because if we want our educators to succeed with authentic assessments, we must think about, okay, the things that perhaps we all hate doing.

Judith and I have been in conferences in the UK and in Australia here in the southern hemisphere as well. And I think I’ve lost count how many times I’ve heard the complaint about, “I’ve got marking to do.” Right. So we must work on that and remove that huge barrier, I think. And then, I think everybody is a supporter of authentic assessment, and we need to focus on removing that particular barrier.

And just talking about marking and Danielle has been talking about Christmas presents. So I think yeah. So we do have when we were coming on to this session we wanted to surprise and, you know, a little bit and you talked about the manual grading of the free text communication piece, following on from the data analytics project. It’s in the email and in speech formats.

So I’m just really happy to report that we are now enabling educators to be able to upload the rubric they have or making changes to an existing rubric for particular pieces of communication type of work. And our marking engine will take that and—AI powered—and will be able to grade the students work. So very happy to let you know this, Danielle, tonight.

And so this not only applies to the written free text communication, but also to the speech, the pitch-type of tasks you talked about as well. And also, I think you mentioned about the integration between AccountingPod and Power BI. I’m also happy to report tonight that we just went live with Excel online. So, essentially, from AccountingPod, connecting to the students Office 365 account. And they will be able to use the magic request feedback and working on the browser and doing basic, I guess, some of the basic Excel stuff. And really so that experience is live as well. And, I’m not too sure about the larger files. I think those you still need to take that offline to desktop, but I think it’s a really great progress here.

Danielle: Yeah, just to explain how useful that Excel online will be. If you contrast what we were able to do here with AccountingPod to how traditionally we would have marked this sort of content. So, previously, for example, I would have had students working away separately in Xero or Excel and then taking some files and uploading those files to our VLE which is Canvas, and in honesty, the risk is that those files could have come from anywhere. Anybody could have done that work. There’s you know, I have no control over who actually prepared the files that the student uploads and then gives to me for marking.

So the idea that actually we could not only have part of the integration to Power BI so that you can see, you know, student B is uploading for me, and you can actually see that student B on their version of Excel and on their version of Power BI actually did that work. That’s a huge game changer for how most of us are marking and how certainly I was marking a number of years ago. So just to flag how important that is to know from an educator part of it. Thank you.

Toby: Thanks, Danielle. Suresh, are you back? The thing that you said that really sort of piqued my interest was this idea that AI is going to be the saviour of accounting education. So just follow up on that point for us.

Suresh: Yeah. So we’ve seen that jump that Danielle also talked about to Power BI that we couldn’t mark at scale those assessment. Now imagine the world we now live in. We can press a button and automate the pure accounting functions. But we’ve had a narrative turn. There’s a massive narrative turn. We have to do climate disclosure. We have to do ESG for some of our clients, or work in that context. We have basically CSRD and to make it even more complex with the tariff wars, prior to that, we had cross-border adjustment mechanisms.

So can you imagine how we get our mind around that, let’s call it these narrative pieces that now are interweaved with our traditional accounting knowledge. Some of this is regulatory reporting, some of this is feel good, etc. But how do we take that into account? How do we take the annual reports, which have more and more narrative, or quarterly reports and why I’m saying AI is the saviour: we’ve all woken up and got excited with AI with large language models.

Large language models help us, just like XingDong was talking about in the assessments; we can actually use large language models to consume and articulate thinking around this reporting. Now the reasoning, the reporting, etc. if I’m conveying it as an accounting student or as an educator, it’s very tough to follow because it’s not numbers anymore, it’s narrative.

And this is why I do believe our profession has changed. That’s new. AI is new to help us mark assessments, and we shouldn’t forget that we’ve barely had time to deal with existing assessments that the new assessments… like imagine setting a question. Can you articulate the challenges of cross-border adjustments and tariffs? That would be almost impossible to follow on paper, but we could ask a large language model to explain to us why the two aspects I’m talking about are working against each other. We could also analyse students’ thinking around them because we’re analysing narratives.

And just to take it out a little bit further, we could have the learner present and through facial gesturing and hands and the pitch in the voice, we can understand another dimension. Or we can analyze an existing CEO or CFO presenting and understand where they may be potential issues.

So all I just wanted to articulate—we’ve not had a chance as educators to develop new assessments in these areas. And yet how can we even mark them, because we’re dealing with lots of narrative and lots of convoluted thinking. So that’s really, I think in the few minutes what I want you to articulate, but happy Toby, for you to dig deeper in any of what I’ve said.

Toby: No, I think I think that is, very interesting. I suppose my my question is, how do we get around the fact that is AI producing assessments and then AI is marking those assessments, you know, how do you actually dig underneath that and ensure…

XingDong: Suresh, I could make a point here.

Suresh: Absolutely.

XingDong: So Toby, just thinking about from an authentic assessment perspective, we must evaluate the interaction between the human and AI. And there’s the approach we’re taking, in terms of learners or working professionals using AI in the accounting context to accomplish tasks. Right. So, we’ve taken that approach. Essentially it’s very similar to how we’re assessing how people are using tools like cloud accounting or data analytics.

And just a quick issue, Suresh, before I pass on to you, the second point I would want to make is that we have to think about the… I always talk about Sir Ken Robinson, you know, the Ted Talk guy who talked about how schools are killing our kids’ creativity. And education grades. Great thinker in education. So he talks about the word “education” and where it’s from. The Latin word he’d use is essentially drawing things out of learners, drawing things out of people. Right.

So if we put that in the context of AI, we must use the tool and AccountingPod is to draw things out of people or give scenarios to students and see their response and value-added, and their response to dynamic scenarios and cases. So that’s a couple of points I want to make here. Suresh, I’ll pass on to you.

Suresh: Yeah. So the thing I wanted to say is that the way in which we ask our students to answer these questions is with a totally different depth. For instance, just as we would defend a master’s thesis or a project assessment or an assessment makes total sense. And that way we dig deeper into the authenticity.

So we have the student’s voice, we have their presentation materials, we have the video essentially. And all of that can be used for us as educators to learn from. And also to discern differences between what is an HD style of presentation or project and what is something that might be mimicked from elsewhere.

So I think that’s the key. Think multimodal and think that these type of platforms, certainly particularly the ones that we’re working on, they are multimodal. because we’re at a stage where we can… this is not theory anymore. We can actually mark emails, as you’ve indicated, and we can mark reports as well, essentially. And we can learn about personality of our learners through the words that they choose and what they’re thinking at the moment they answered the assessment.

Toby: That’s great, Suresh. Thank you. I mean it’s exciting, enlightening and appalling and frightening that we can do all of those things. But, yeah, I mean, you know, interesting times ahead.

Q&A session

Toby: Look, time is creeping on. XingDong, did you want to add anything more to that?

XingDong: Toby, I think I think we have made quite a lot of points. I think it’s probably a good time to just get some questions and have the conversation.

Toby: Anybody got any questions?

Danielle: Toby, I guess if I may, just when I think about being an educator and first engaging, I remember struggling to get my head around what might this look like? And what might, you know, what are AccountingPod actually offering. And, you know, how could I engage with that? And how much would I need to bring to that versus how much do they already have there? And, you know, in honesty, we sorted a lot of that within a half an hour conversation with Judith and XingDong.

Actually, a lot of the materials really helpfully are, as they say, a library of materials tried and tested. So I was able to have a really good look in and have a really good poke around of, “Okay, this is what you’re offering, and is this actually going to meet what I need—is it going to do what I need to do?” And I think that conversation and that back and forth was so much more than you can ever get out of a presentation and things like that.

So I suppose just to say that and just to say I’m happy to chat with folks as well at any stage, about how we’ve… actually, in fairness, I’m giving you the example of what we first did with AccountingPod. We actually now I also have other things with, you know, our accounting for our Excel and Xero for year one is also through AccountingPod. We are bringing it into our Masters qualification this year, something akin to what we do with the year threes. And also we’re looking at a capstone project as well. I think we’ve got some questions coming through there. I’ll stop speaking, Toby.

Toby: Well you’re probably better off answering those than I am. So Rhiannon is asking about how this works with larger cohorts, and she’s talking specifically about professional exams. How scalable is the platform?

Danielle: So I suppose there will be things that will work extremely well, you know, because all of that automated feedback for the students in a way that you could not dream of giving those individuals feedback will work extremely well. The marking obviously aspect of it will work extremely well for you as well.

Danielle: One of the things that I didn’t mention is the analytics that I actually have as well. So right through, you know, so I’m in week two of the teaching and I can double check and say, “Okay, who has not only maybe not been in the class, but maybe not engaged with AccountingPod as yet,” so I can drop them a little quick email to say hi. So you’ve not engaged. What’s happening? I think those sorts of things would be particularly helpful. I’ll let XingDong and Judith speak to the more technical logistics for a larger cohort.

XingDong: Yes. So absolutely. We’re built for scale. I think, at this stage, we’re doing about 40,000 completions a year. Yeah. So no problem with large cohorts. In fact, when we started, we started with very large cohorts. So it was about I think if you think about in the Australian international student context, we have 1500 students in the introductory level accounting class. Right. So, yeah. So we’re very used to large cohorts and we do work with professional bodies. I think, if you’re referring to professional exams, yeah. So all of those things, and we’d love to do more. So if you have particular referrals, please send me an email. I’m happy to follow up and talk about how we can help. Yeah.

Toby: So hopefully that answers that question, Rhiannon. And Toby, would you like me to answer the next one? Just in case everybody hasn’t got access to the chat, Vicky McCafferty is asking about audit assurance modules at level six.

Toby: Oh, look, you’re here. Ask your question. What’s what specifically do you think the challenge is?

Vicky: Authenticity. I want the students to, you know the phrase, “learning by doing”. I want them to actually perform audit tasks to actually understand, because just teaching theory, it doesn’t teach auditing, does it? It can teach them to pass an exam, but it doesn’t teach them how to think about what they need to do, what the risks are, and things like that.

So I’m very new to academia. I was in practice for 26 years. So 18 months into academia, now. But I’m just constantly trying to think of ways to make it more authentic. I know with accounting it’s easier, isn’t it, because you can do some management accounting or prepare some financial statements, but the actual auditing side of it.

XingDong: So, Suresh and I have worked on, in particular in auditing, in the context of auditing, auditing blockchain and carbon credits stored on blockchain. So, you know, really I call that the three birds with one stone. You can tick off blockchain and tick off ESG carbon credits and also auditing as well. So I think Suresh, if you’re still on the call, would you like to talk a bit more about that particular case?

Suresh: Yeah. So we’ve taken a real world case. We’ve basically gone through the lineage of where they can be saving on carbon carbon. And because the carbon credits are voluntary, we actually allowed them to be tracked using blockchain. And so that, if at some time in the future, those carbon credits are made available to someone else, there’s no double counting. An auditor can come in and basically also audit the carbon credits as such. And we also account for that in ESG reporting and other forms of reporting.

But essentially it’s… I think there’s a number of modules now in that space. And I think it’s something the XingDong can perhaps provides, if it’s highly relevant, access to. But it is audit at the leading edge. And we believe it was easier for us to help educators start there than going back in history, because there’s a lot of traditional audit that’s been done.

Toby: Can I just… yeah, because we’re running out of time, and I want to make sure we answer some of the questions. Helen, are you around? Do you want to ask your question?

Helen: I am, yeah, can you hear me? Hello, and thank you. It’s been really interesting. My questions really directed to Danielle, and I was just wondering, how do you, when you’ve integrated it throughout your program, have you set up particular modules? Have you created a new module that’s focused around this? Have you embedded it into an existing module? I was just wondering at that level how you’ve done it.

Danielle: So, so in our first year module that was an existing accounting information systems foundational module. It has an exemption attached to it. So that obviously meant that we’re tied to a certain percentage exam. So the AccountingPod element of what we do has always been quite a big chunky part of the module. But it’s worth, I think, about 20% of the module. And that’s just part of continuous assessment.

On the other module, the third year module, we probably would have a lot more flexibility, which I haven’t really taken on. That part of that is also because our students, believe it or not, it sounds strange, but our students tell us they like exams even on, you know, where I could send them off to do group work or, you know, or a continuous assessment. They come back to us and say, “But actually we’re used to exams. It’s what we do.”

So in honesty, even in a module like this, where you could absolutely have them going off and doing a case study or something like that, we maintained the exam and we tried to do a halfway house because we give them the same case study two weeks out, and then they come in and answer questions in the exam. And it just gives us controllability about the exam. But it also gives them a sense that what they need to know is bounded. I think that’s what they sometimes struggle with when it’s an assignment, especially an assignment for like a really high proportion of the marks that they’re just not used to in accounting.

You know, so I think it’s been a smaller proportion of the marks rather than the full mark. But if you were doing more with them, you could scale it up. The other part of that module is contemporary technologies. So that’s where we go through—we have a couple of weeks with multiple guest lectures and things like that. So that’s the other part of it.

But then this AccountingPod part of it, that’s fully linked to all of the teaching of the specific technologies. And again, those are technologies that the employers asked for. And we give them, yeah. So and it works out at about 30% of the module for that. So that’s fully, continuous assessment. And they get the results within a week as well. So they have all of that before they go into the exam. Most of the rest of it is done through a computer lab and the exam.

Because that’s we have some stuff around Gen AI, plus a case study—it’s an unseen case study—for the other part of it. Again, just trying to be ahead of some of those conversations around let’s not give them something they can stick through AI. So no, always part of an existing module. Same thing for the Masters as well. It’s a broader module around contemporary technologies. And then the AccountingPod is the articulation or the use of some of those technologies that we’re talking about.

Helen: Thank you.

Toby: I think the next questions for me as well, Yes, Danielle, thank you for that. I mean, it’s wonderful hearing all of this. Whenever we try and bring anything that is sort of vaguely external within our assessment scheme, the centre, the powers that be always say, “Well, where is it hosted? Is it in the US? Is it in Europe? What about GDPR? How do we maintain the security? What if a student appeals against their grade in 2 or 3 year time? How confident are you that this external material will continue to exist?” And all of that? And I’m very often left sort of like speechless in terms of being able to answer that. Did you have those kinds of questions and how did you deal with them?

Danielle: So not so much around the 2 or 3 years from now on an appeal point of view. But absolutely. I mean, we had quite a few questions around, particularly the API link that needed to happen into students 365 accounts. The good thing that we had was that actually XingDong and Judith have seen that all before. So they were able to provide a lot of the information upfront that the university requested.

As ever in the university, none of this happened quickly. So the advice I would have to anybody is, if you’re thinking about doing this for your university processes, you probably need to be quite well ahead of time in terms of making sure you can get the right approvals and things like that. But, you know, they worked through it. At the end of the day, you know, I think it’s an established product. It’s in other universities. Richard, if you’re having the conversation, if you want to cite Queen’s University as an example of an institution in the UK that’s using it by all means. I don’t think I had that at the time.

This is a few years ago since we started in this, you know, so we have been using that, you know, and in terms of, you know, in terms of the question about, you know, a number of years down the line. Well, you know, ultimately, the data is there, the data exists. I’ve been able to download it. You can get a huge amount. I can download the data, I can then use it and calculate the marks and things like that. So I have that data sitting there so that does belong to me. Once it comes out of AccountingPod, I have that and I could go back over that. Yep.

Richard: Thank you very much. That’s very helpful.

Toby: Thanks.

Closing remarks

Toby: Thanks, everybody. Look, we’re right on 11:30. I’d like to start on time. Finish on time. So I’m going to say thank you very much to Suresh, XingDong, Judith and Danielle, thank you so much for your time and for your contributions and for being part of this conversation and community. Do please stay in touch. And as I say, sign up to us on LinkedIn, join our newsletter. Keep in touch. It’s good to be with you all. Thanks so much.