The Accounting Technology Lab

Brian F. Tankersley, CPA.CITP, CGMA 00:00
Welcome to the Accounting Technology Lab, brought to you by CPA Practice Advisor, with your hosts Randy Johnston and Brian Tankersley.

Randy Johnston 00:09
Welcome to the Accounting Technology Lab. I'm Randy Johnston with my co-host Brian Tankersley, and we are so pleased today to have as a guest Alexis Kingsbury. Now you've heard us refer to Alexis's book, "A Cruel Attentions, in other podcasts, and you know I don't know which person we're interviewing today exactly, whether it's Alexis or whether it's you know Buzz Clickman or Sonny Rapport or one of the other agents that were in that book. You know, 11 agents that he created. But Alexis, would you mind giving our listeners and a little background on yourself?

Speaker 1 00:49
Sure. And I promise, this is the real human version of me. You're not getting the AI, getting an AI agent. Is this

Randy Johnston 00:55
in an avatar or something?

Speaker 1 00:59
I don't know how I would disprove whether it is or not. So, so you have to sort of trust my personal brand that that's not a thing I do. But yeah, no. As you say, I you know I wrote a cruel intentions, and I suppose it's probably worth me explaining why. Why did I do this? Why did I create the world's first 100% AI accountancy firm with 11 AI team members, each with their own name and role and personalities, and it's a fair question. And I suppose it probably helps to kind of go back to you know what's been my background and where'd this come up for me. So I suppose if I was to summarize everything that I've done over my career, you know, over two decades, more than two decades, it's been relating to management science, or which was essentially the name of the degree that you know that I took it when I was at university at college. But essentially, that's around how do people work, how do organizations work, how do teams work, and how do you optimize performance across people and systems and all this kind of stuff. And in terms of my journey, that's included serial entrepreneurship. So I've set up and built multiple businesses. I have also been a management consultant working with other organizations on how they operate, particularly specializing in business operations and people and processes. So particularly the process part across teams. So how do you get complex processes that span across an entire organization, and make sure the handoffs and so on work well, make them as efficient as possible, make sure that they increase value to to the customer, and so on, and you know, building that both within teams, but also then scaling across large international organizations, and then some of my businesses have been in software, so I have three software product businesses presently. And one of the journeys that I had with those was, you know, building my own businesses, but particularly building those businesses to run without me, so that I could, you know, have freedom to work on the things that I enjoyed and be able to to have a greater impact. One of those software businesses, Air Manual, which is all about documenting your processes and your onboarding and so on, which was particularly solving a problem that we ourselves had as we were building one of our other software businesses, turned out to be a really good fit in the accountancy industry. So I ended up doing a load of work with accountancy firms that then got me pulled into accountancy events. I've always enjoyed speaking at events, so I ended up speaking at these events, and it was there that I really started to see the struggle that people were having conceptually around AI, and particularly what I started to see last year. So when we call in 2025, I was seeing these two camps forming around AI. One was saying it's amazing, incredible capability, and it's going to you know might disrupt the entire industry, indeed society, humanity might take all of our jobs. That's if it doesn't kill us first, and so that was kind of the narrative in one camp, and then increasingly saw this other camp that form, which says actually all of that is hype just to boost investments in these firms because they need to spend hundreds of billions, and therefore they need to scare us all to be able to to get that kind of money. Actually, there's loads of studies that say 95% of of AI pilots fail. Most people overreport the time savings they get. Most enterprises and organizations are failing to capture any benefit from using these AI. The AI itself is massively under priced versus the actual cost, and therefore there isn't even ROI here. And so I saw this divergence of these two views, and the problem is that I saw that when I looked at both of them, there's truth to both. Like my own experience of using AI shows that there is incredible capability, and most people have had some experience where they go, "Wow, that was incredible. But then equally, we've all had the experience where AI can do something very stupid that makes you go, "I'm going to have." Check everything. Would it have just been easier to do myself?

Speaker 1 05:03
And it was that realization where I particularly heard that phrase of "Oh, it's just easier just to do it myself rather than get the AI to do it and check it, which rang this big echo for me with what I've been doing for the last few decades with humans, because the number of times myself as a leader, but managers and leaders that I work with who say the same thing, but about team members. I.e. well, I would delegate that, but it'll just be easier if I do it myself. And so, as soon as I start to see that and start to look at this problem, I said, well, maybe maybe I can model this. Maybe I can take this to the extreme and see how where does AI succeed? Where does it break? And as a result, what does that mean for all of us and the future of work? And particularly knowing that some of the barriers that people are encountering when they're using AI were ones that I already saw as solved problems outside of AI with human teams. And so that was my starting point for why I wanted to essentially turn to the back page of what does the world look like when AI is hugely capable and can do these things, and what does that mean for accountancy firms, and what does it mean for organizations at large, and so I ran this crazy experiment using Agentic AI, and we can go more into what that was and so on. But it was by going through that experience that I then concluded actually there's some really important lessons learned here that are bigger than just oh yes this is helpful for me delivering a talk and I concluded I need to write a book on this so that I can take people through both the intellectual journey but also the emotional one because I was very worried that many people would go halfway along that journey, come to some conclusions, not continue along the path, which you know would be a logical thing to do. Whereas I'd kind of irrationally continue to take the experiment further, and I wanted to share the outcomes of that.

Randy Johnston 06:56
Well, see, that makes great sense. So, as just a reminder to our listeners, you know, we've thought about AI in three tiers, with the frontier models kind of driving the productivity. So again, we are probably less concerned about which frontier model you're using. You know whether it's ChatGPT or Claude or Copilot or Gemini or Perplex. We don't care on that. But then the second level was where you are buying products that already have AI integrated and somewhat tested by major publishers, you know that helps with accuracy and integration, and there's some value to that. But we believe that this third level that you operated in, Alexis, the use of model context protocols (MCPs) plus agents was where AI could have the most productivity, and you know your illustrations with the controls that you had to put in place and the errors made and the logs and so forth were, you know, fascinating. You know, just in terms of the experience, because I think you're right. Many start experimenting and get discouraged and stop, and we don't think that's a good strategy. So you know, I I know. In fact, I was musing while you were saying it because, or writing it, I guess I would say, because you know you were talking about how you were really not well while trying to do this experiment, and you know pick it up and put it down, but then you'd see the results and so forth. So, just if you don't mind, for our listeners' benefit, talk us through how you started building the agents and how you kept them in control and how you decided on personalities. I don't want to go into too many different directions, but you know, I'm just trying to get the arc of okay. I decided to do this experiment. That's what you've set up to this point, and here was the outcome. And the outcome was bloody thing works.

Speaker 1 08:52
Yes. So it was about August September 2025 when I had the this crazy concept of I'm going to create the world's first 100% AI accountancy firm just to see what it looks like and how it breaks. And partly at that stage, I had intended it as part parody, part experiment. Like I was partly wanted to show people that yes, you can create a website, yes, you can create all this front, but like there's nothing there behind it. Like in it feels very Wizard of Oz nature, where it's like you know, pay no attention to the man behind the curtain. And partly, I also wanted to see what would happen if I essentially empowered AI to try and make decisions about setting up this firm and who's the ideal customer and all this sort of stuff, just to see like what happens when you give it more autonomy and empowerment, rather than just use it as a tool to do a thing that you've told it to do. So initially, I set up a series of custom GPTs, which now feels like a very old. Despite that was still a thing, like probably a thing only a year ago, now already feels incredibly old. And for so for those that aren't familiar or can't remember what those were, essentially it was the ability to create to curate a set of files and instructions that essentially just would be your default starting point for a particular custom GPT, and so that you could invoke this custom GPT by picking it when you start your chat, but it then means anything that you're then asking it has got the context that was already built. And so, what I did to start that off was essentially told at the time that I was using ChatGPT, and I said, "Here's the idea: we've got this accountancy film we're creating called Accrual Intentions, which it loved the pun, by the way, and explained to me that this is a joke because of this, and it's kind of, yep, no, you, I, that was my joke that you've now told me as a joke. Like, well done, and I basically tried as much as I could to say, right, but now you run with it. Now you're building this. You're just making these decisions, and as much as possible, I was trying to not put my fingerprints on it. Like one of my backgrounds as a consultant is as a facilitator, where we talk about you know you don't want your fingerprints over the outputs. You're trying to coach and facilitate towards the result rather than just tell people here's the answer. And so that's what I tried to do wherever possible. However, I soon found that was easier said than done, particularly because, for example, one of the first things I got it to do was right. I want you to assemble your team. You decide what roles you want. Give I want you to give them names and person. Like you know, at that point, said I want to give them names and create. I want you to create pictures for each of them, and or at least describe the pictures for each of them, so that we can then create that with a separate tool or a separate prompt. And very quickly identified a problem, which was that it very much saw my accountancy firm as consisting of basically only white males wearing glasses. And I felt quite quickly that yeah, I don't want fictional or otherwise, I don't want to own a lux that has that problem with diversity. So as a result, that was the first fingerprint I put on it was to go, no, we're going to have a diverse team, and then it responded, and so then the team had incredible diversity of both ethnicity and gender and orientation and so on, which was really interesting and kind of great to see, but also I said, and of like, and I want you know diversity and of personality because one of the things that I've experienced as a consultant and as a business owner is actually diversity of perspectives is very important, and that if you want a really good decision made, you don't want two people who think the same.

Speaker 1 12:40
You want different perspectives, and so that was one of the reasons that I wanted to create this team with these team names and so on. Is I believed my hypothesis was that actually by having these different lenses and viewpoints, you could arrive at better results. And as part of a process that I knew I'd be building later, I'd be able to inject the right lens at the right stage at the right review point or whatever. If I had this mental model, and so and

Randy Johnston 13:06
you know on that point, sorry to step over you there, but just as an example, you know speaking in French I thought was just a hilarious twist on all this. So when you were talking about the diversity and the viewpoints, you know it became so clear so quickly that the agents were in fact developing these personalities, just like you had, you know, the culture, Holly culture, and you know, Reginald Rules and so forth. These different agents with the personalities. It was pretty clear to me in reading your book that you were probably enjoying it evolve as much as I enjoyed reading the evolution on these things, but your point here, Alexis, that having different viewpoints gets better results, I think, is really critical. And you're right, you know, this profession really around the world frequently has you know older white male associated with it, and there's some wonderful accountants of all sorts of ethnicities and backgrounds that you know we need to leverage, and likewise we need to leverage that in our AI. I think so. Sorry to interrupt your flow there, but just the Z answer on this was so much different.

Speaker 1 14:27
Yes, indeed, and like you mentioned, the French member of the team, so Debbie Double Entry, our technical accounting lead, which and it makes me slightly blush every time I say her name. The AI did pick her name. Obviously, it was playing on the fact that I had given it the pun of accrual intentions and just the way that I was with it and a very playful. Then it decided, right, we're going playful with all the names. Hence, Buzz Klickman, our head of marketing, Reginald Rawls, head of compliance, etc. But yes, as part of that increased diversity, we ended up with Kate. Casey Caremore, who happens to be Australian, doesn't really come up in the in the book. We've got Cy Billlocks, who's head of security, who happens to be Scottish, and as you say, Debbie, who happens to be French. And the way that it came out as a result in the dialog with her was this sort of use of Z heavily. So this is what I think, and so on, to create that in written text, right? Because it wasn't the conversations I was having with it, and the team members were having together was not verbal, you know, or voice. It was written, so you had this sort of affectation that went through, which you're quite right. I enjoyed immensely because for me, it was I wanted it to be funny, partly just to make it entertaining, particularly when parts of the experiment were quite bleak, but also because my intent was that I was going to talk on stage about what did I learn. I didn't think that I was going to be turning into a book that came much later. But yeah, I mean, I've I've always been a playful soul, and so I liked that concept. But certainly there was a very serious point, which is around if you want, one might argue that a form of superintelligence already exists from AI models. It's just that you don't get it to speak with one voice. Like if you any question you can pose to the factotum in the form of ChatGPT or Claude and get an answer, whether or not you agree with that answer, whether or not other people agree with that answer, or can prove it or disprove it or whatever is then a separate matter. But you can get an answer, and my experience has been actually what you really want is different lenses on an answer so that you can understand a position or a situation from different perspectives, and that allows you to then get a much better result. And so that was my starting point was back in that August September very turn based very much you know at one point I was even copying and pasting responses between custom GPTs to support them in having a conversation and then it wasn't until I start to really experiment with Claude Code and then later Claude Cowork and then much later OpenAI's Codex and so on was then when I was able to really get it popping because I could have them have those discussions without me having to do a load of copy pasting, but also as you say, it was agentic, so I could get them to actually do things, build files, create documents, edit documents, access tools, and so on. And that was what happened in February, which, as you mentioned, I was unwell. I had a cold, and so I was in. I was kind of on my bed, feeling unwell enough to do proper work, but well enough to have a play at an experiment that I had previously sort of temporarily parked. And so as a result, yeah. Then over the subsequent few weeks, I then went very extreme with it and didn't sleep much during that process, partly through excitement, partly through terror, and yeah. And the book was essentially telling that, particularly that story in detail of what happened and what I learned during that.

Randy Johnston 17:52
And you know, to me, what's also fascinating is the logs of the errors, the things that go particularly wrong. And of course, in today's context, as we've watched now the evolution of Claude Code and then cowork and so on. You know, I think it was just this past week or so that OpenAI wound up overtaking Anthropic again in terms of developer spend. So it's been a beautiful horse race to watch these go back and forth as we would see it, but you know this idea that your agents, in effect, kind of worked with each other and the unpopular methodology right now, the swarm and the attacks that OpenAI has been reported as making against Hugging Face and others do does give you a little bit of a pause, but again, Brian and I both tend to be a little on the optimistic side when it comes to these things. So we're not Pollyannaish about it, but we're also quite committed to the accuracy of accounting.

Speaker 1 18:58
Yeah.

Randy Johnston 18:59
So Brian, I think you got a an insight there.

Brian F. Tankersley, CPA.CITP, CGMA 19:02
So Alexis, the I guess you know there are things that large language models and AI are particularly is particularly good at, and there are things that it is particularly bad at. And you know, in some of the v4 models, there were problems with it doing math and other things like that, and the skills caught up. But what you know when you were when your team was forming and developing its descriptions and everything, what were the the pieces of the puzzle that were the most difficult to tune and get right in here, as opposed to you know the things that it just did natively inherently well?

Speaker 1 19:39
Yeah, so I'd say that the thing that exactly you're exactly right. The thing that you observe as you use the models over time, you spot that there are some things that starts off terrible at and then seems to get better, and so then you start questioning. Oh well, actually, does any problem survive? Is it just a matter of time before that? Gets overcome. I think the mathematics one is a nice, is an interesting example because even at the time when you know GPT three and four etc. were being criticized for oh well it can't do maths, the researchers, the people at OpenAI etc. were showing or highlighting, but we haven't given it a calculator. Like its ability to do maths is a derived function. It's an emergent property of it having learned language, and in fact, we know we hadn't purposefully trained it on different languages, and yet it seems to have developed that too. And so, I think I'm hearing less of it now. But over the last few years, there was a lot of talk of this emergent properties that that kind of come out from these models, and of course, now we're seeing we're like with ChatGPT six with Astra. I've seen some people joke about how it just seems to be a sort of AI front end to Python, where basically it's like you ask it to do a thing, and it then goes and works out how to use a Python script. You know, for those less familiar, a bit of code on your computer to do a thing, and someone criticizing it for that, like I'd argue, it's like brilliant because you don't want it to attempt maths through emergent language. You want it say, okay, you want me to do this complex formula. Well, let me write a piece of code that will calculate a formula and then run that and give you the response. And that's actually hugely valuable. And so, with the organizations I've been working with, and indeed in my own businesses, often I'm looking to use AI to build something that is deterministic, i.e. through calculation or whatever, rather than build probabilistic, as in prediction of next token or next word, as the AI into my workflow, because that's unpredictable by its very nature. Whereas, if I use AI to build something that's semantic and predictable, I can then get these better results. And so, often that, as a shape of the work that I was doing, definitely endured. And if anything, that it makes me think about human systems, human organizations, where often you start off with in any new business with a lot of experimentation, you don't know what's going to work, so you try loads of things. You throw things against the wall. You know, a big part of my role in my software organizations has been speaking to potential customers, asking them questions, understanding the problem that they have, and then over time, I get to a point where I can test the description of a problem based on what I've heard from numerous conversations, to the point that I can describe it better than they can, and then they go, "That's exactly it. That's the problem I'm having. And over time, I get to a point where I can describe the solution in a way that makes sense, and the lights go on. And that's that experimental journey. However, if I then go right, I need a salesperson now because we've worked this out, and I go right now. You do that. It's incredibly inefficient for them to now do that same level of experimentation to come to that result. And equally, they might not consistently use the right wording, which gets a result. And so, in my organizations in consulting, I've done what we typically do is document that as a process.

Speaker 1 22:57
We document that as how do you now have a discovery call with a customer to write to ask the most appropriate questions to confirm the most important details and to describe how whether our product is a good fit for them and how it would be and all that kind of stuff and do that in a consistent way that is a good efficient use of time for that client and so you're codifying it into a process still operated in that case by a human, but it's you know you're codifying it as that, and it becomes more deterministic. Which means that if someone's not getting as good sales as someone else, then you can go and look, and you can go and compare those things rather than just go well, you know, it must just be the person. And I think when you apply that to AI, it means that you say, well, rather than just saying, hey ChatGPT, can you produce a set of accounts for this client? You've got access to their, you know, their QuickBooks or their zero data. Instead, you go, no, I need a process and some templates and some human checklists so that I know how this information is going to show up. I don't have to check every single formula every single time that there is stuff that I can predict, and therefore I can make these decisions, and so that was one of the journeys that I found was that actually, in many ways, you're using AI to build a thing, not necessarily to use AI all the way through the process. But I think, and partly that's because something that I have seen as a consistent gap throughout all of the models from 2022 to 2026, and I can't see it really changing. Is a risk, an error rate, which doesn't that improves in some areas, but it's always there. And a great example of this just from yesterday. So one of our software products that we have allows you to you know manage all your tasks, your organizational context in one place, and point your AI to it. And so I'm having a conversation with my AI. In this case, it was Claude. You know, most up-to-date model, pointing at that and saying, right, help me complete this task, and it helps me think through. A series of steps and move things and update the documentation and move things around and do all of that and does it brilliantly. And then its final thing is okay. What we now need to do is we need to add a new task to do X. It's like great, do that, and it does it and then gives me the link, and I click the link and the link is broken. Now because my software product is in the background, I'm thinking, oh, it must be a bug. Our side let me go and investigate, have a conversation with AI. Turns out, my product with an API gave it the exact link to give to the user, and it then just slightly changed it, and it gave it to me, which broke it. Anyway, right, you were genius for all the hard stuff, and then did a really stupid thing at the end, and in that situation, you might go, "Oh, it doesn't matter. You could have found it yourself anywhere. You could just tell it link didn't work, and it would probably diagnose it and fix it. It's like, but that means like that's a checkable thing. It was, you know, it's is the link broken and goes to the right place, or you know, or not, or whatever. There are loads of things in the work that it did that aren't checkable that I'm looking at, going that was brilliant, but there's probably errors in there too.

Speaker 1 26:08
And I think that's the problem I see with a lot of people is they'll use AI to do something that they're incredibly skilled at and say, oh, look at that, it's stupid, it makes mistakes, and then use that same AI to do something they don't have expertise, and go look how brilliant it is, and assume that its error rate was different, and that, and I think that's the fundamental problem at the heart of at least large language models and generative AI. But you can build around that with processes, with checks, with building deterministic parts, and so on that allows you to do it, so that's where I end up. Is like there's value, but also risk, and and and it's all about how do you build to maximize value and mitigate the risk.

Randy Johnston 26:51
And as you're laying that out, I just consider the mistakes that I make when I'm tired, and it's like what was I thinking? So you know, if I do something late at night and pick it up again in the morning and say, "Oh my, how was I able to do that? One of my favorite managers' phrases, which is is frequently used nowadays, less frequently used maybe over the prior decade, but there's a limit to how smart people can be and no limit to how dumb they can become, and you know, as it turns out, I think that's what you just said about the AIs-brilliant in many things-and then they make a simple mistake rather at the end that's checkable. And you know, this whole idea that the AIs can do brilliant things is the promise that I see. And whether it's in accounting or biological sciences, or wherever they wind up going, the promise has been so great. And you know, we aren't going to take you down the path of AGI. But I've been watching AGI for decades. At this point, there's going to be a point in time where these AIs can do things that none of us can really imagine today. I think it's believed right now that the IQs of most AIs are operating in the you know 190 range, and I think 150 range was where they were a year ago. So you know that's smarter than most people, as it turns out, and this idea that they can solve strategic problems, that they can solve tactical problems, I think it's clear that the products can help us, and the agents can do it better in many cases than our people can.

Speaker 1 28:40
Yes, and I think to some extent you could argue that that's been true for a long time, and yet still doesn't necessarily move the needle. So, to give an example, so more than 20 years ago, I remember I so I had a job. I was initially brought in as a paternity cover to do IT support for a contact center for DHL for a logistics company, and really interesting job. But I remember that there was a particular person in the same department where their job was to do this particular thing, where there's a series of reports that come off a system, and so that data is in the right place to be then picked up by another system every day, they had to basically move the files into the right folder. Now, one particular weekend they weren't available, and so I was asked, "Would I come in on the Saturday from eight a.m. to 2p.m. to do that job, and that it would take largely that amount of time for me to do. Good news, you're on time and a half to do it. And so I was like, "Oh, okay, great. And then I started doing it and found it incredibly dull, and thought there must be a way to do this more efficiently. So I start looking online and I learn about these things called batch scripts, which I'd never heard of before. Which basically allows you to write some code, which I'd never written before, and instead I was basically looking online and getting the guidance on how to do it, which would allow me to define some parameters and say if it's this, then put it in this folder, and if it's this, and this so on, and then I found actually it could create the folders itself and so on, and I got to a point where for each of these eight reports or eight sets of things. I could run this individual batch script, and then I remember doing all of that, and then going, wait, I can just create a batch script that then calls all eight. And so the following Saturday, because I think I had to do it for like two or three weeks while this person was on holiday. All I had to do, like at 8o 8o one, I ran one batch script, and it did all of it, and that was all of the time. Right, that was the six-hour work that I was brought in to do. I was able to do in two minutes. Now, two things: one, those automations existed for probably even a decade before I came and did that particular task, and there were still many opportunities to automate and improve all sorts of parts of that business. That some were done, some are not done, etc. Just because you can do something doesn't mean that it gets any focus or priority and attention to do it. But also note that maintaining those batch scripts suddenly requires a different skill set, and so on. Then that doesn't come without its own problems. But the other thing is then to go, well, what did I then do with the time? Because one of the concerns people have about AI taking, you know, doing this work and so on is, oh, well, it then takes jobs, etc. But I didn't then just sit there going, oh no, I'm going to be made redundant from this. Instead, I looked at how can I add more value, and started doing other proactive things and getting involved in project management and building a new intranet site and all these sorts of things that weren't previously possible or cost-effective for me to do. And so I always think about that when I'm thinking about AI and how we can automate and do all these things. Is in many cases the ability to automate, the ability to use Python scripts on a computer has existed for a long amount of time, and even getting the guidance and the insight on how to do it has existed. It just comes down to human attention. It comes down to what we believe is worth solving and how we shine a spotlight on it, and typically, once you make something automated or solved, it isn't true to say, "Oh well, and that was the total amount of work that would ever need doing.

Speaker 1 32:28
Jevons' paradox applies, where you essentially go, "Okay, now that is easy to do. There's more work that can be done, or you could apply Parkinson's law, where the work will always swell to fill the time available, and so I think that's what I'm seeing in every industry is where you are getting the opportunity to do as you say. You know, use these high IQ systems to do these things. Ultimately, it's coming from a position of someone shining a spotlight on something, and then working on it. But then it creates more challenge, different challenges, and more work, and that's the very nature of sort of building things. So, whilst yeah,

Randy Johnston 33:07
so Alexis, I appreciate the explanation on the DSL. That's a brilliant example. And I, as you were explaining that, I was considering your earlier consulting insight, where you listened and then summarized, and the aha moment occurred, and to me, that's actually this insight is really the expertise right now that the humans have, that the AIs might lead us to, but currently don't exhibit. Right, a lot of it is factual, but you know this idea of the new idea. If we have the right context, I've watched Brian do this many a times. I'm listening to you saying, "I bet you've done it many a times, and I've listened to many consultants and accounting professionals through the years have that aha moment. And the most common question afterwards, how did you come up with that idea? Well, I don't know. I just kind of listened to you and came up with it. Why do you ask, right? So, so I'm sorry to step on you as you were headed to another point, but that's to me, that is such a high value because if we can stop doing the mundane and have time to think about more interesting and higher value work, that's exactly where AI and agents can take us.

Speaker 1 34:28
Indeed, and you know, to use that that same job, like 70% the normal job, the IT support job, 70% of that job was turning computers off and on again, like or speaking to someone on the phone who tells me about a problem, and I say, "Have you tried turning it off and on again? And in most cases, they go, "Okay, I'll try that. Okay, yes, that's fixed it. I do remember one occasion where the person said, "Yes, that didn't work. I was like, "Oh, that's strange. Went to their desk, spent a full hour trying to diagnose, and eventually turned to them and going, "You did restart the computer, and they're like, "No." It's just that like IT always says that, doesn't it? And so I then restart the computer, and of course it works. Now, aside from whether or not you can trust people and AI, the other thing that you can take from that is 70% of my job was turning computers on and off again. Now technology is good enough that isn't as high a need. Like when was the last time you actually had to restart your computer to fix a problem. It used to be something that most people had to do once a day, once every couple of days, maybe more. But did that job disappear? Of course not. Like, what would 19-year-old Alexis have been doing back then? Well, yeah, it wouldn't be that the job didn't exist. It's just that the nature of the problems I was solving would be different, and the learning opportunities would have been greater. And I think your point there around actually, if you can get rid of the things that we don't enjoy, don't add that much value, etc. you get more time on the things that do add value. And I think there's this assumption, a little bit like when we talk about the IQ piece, there's this assumption that we say, well, when the computers get so good, then what do you need the humans for? But it's actually it's like the humans can improve on top of AI. Like the humans have access to the AI. So if the like hypothetically, let's say the AI gets good enough that it's got an over 200 IQ and it can answer any question better than any human and do so without errors, right? Which you know that's a big statement to get over. But let let's say hypothetically that's been solved. The human can still just ask the AI and get the answer, and then any work and and has got it instantly, and then any work they do on top is then adding value. What I think we're unlikely to see is a time where consumers say resist or don't let a human touch that. I want that directly from the AI. The computer is the only one that I trust, and I worry that the human will make it worse. I think instead we will always be looking for how does the human elevate on top? How does the human take the oh yes we've got this answer from the AI and then make it more relevant to that person's business or their personal situation or whatever. Like some of the things that an AI can't do is care, and at no point, no matter what it says, can you really get mass population believing that the AI properly will care and so on, right? And because it doesn't, and it can't therefore value things and help you prioritize and so on, because it doesn't have an innate desire to do those things. It is just trying to essentially, you know, achieve the goal based on an assumed score that it will get. Yeah, in

Randy Johnston 37:35
fact, this whole issue of being sympathetic is one of the things that drives me crazy about AI. You do not have to stroke my ego. I'm really asking you a question, and you're my assistant. And so, when I get that type of feedback from an AI, Alexis, it's just like, please, no, you're wasting my time and your time trying to make me feel good about your answer, right? So, so you've had now the experience of building these agents, and obviously the long-term experience of building other businesses as well. So now that you've kind of made that journey, what are some of the stumbling blocks that you would caution other people to stay away from? Here's an error I made, or here's a place that you could stumble and fall because we again part of Brian. My job is to try to keep our listeners from making errors that have been made before. I think Brian says it best. We want people to make new and interesting errors.

Speaker 1 38:33
Yeah, good advice, and particularly because I always think that in business there are an infinite number of ways to succeed, and often a shorter number of ways to fail, and so therefore often it's more useful not to study the successes because you've got survivor bias there, but instead to go, what do I know doesn't work, and you need to be careful of. So I completely agree. So I'd say one of the most common ones that people talk about is obviously data security, whether or not the model's being trained on your stuff, and so on. So I'm going to take that as read. That most people have got their head around that, but if you haven't, start make sure that you do. Then I'd say the next thing is, of course, applying your judgment to situations and avoid. It's fine to delegate the doing. Be really careful, delegate the thinking. And I see that both for leaders, but particularly for more junior employees, where they get to a point where they go, "Wow, yeah, it's got this 190 IQ. Therefore, its ideas, its plan, its review is probably more valid than mine. Therefore, I won't get in the way. And I'm seeing lots of examples of people, you know, like a manager asks a junior employee a question, and they get a response back with Claude said, and then copy and paste, and or even worse, without the bit that says Claude said, but quite clearly or entirely written by Claude, and it's okay to be written by Claude, but actually, what's the insights like? What are the key points that the human thought were worthy enough to share and offer? And that delegation of work has become delegation of thinking, and that's you know very risky. And we've all probably seen examples like you know someone's put together a slide deck or a LinkedIn post or whatever where you read it and it's well written because it was done by AI. Except that there's no real meat there. There's no insight. There's no moment where you're like, "Oh, that's a really interesting way of putting that, and that comes, you know, my background consultant that comes away from thinking and asking questions and challenging yourself and looping back and then trying to get it down on a half a sheet of paper before it becomes a 20-slide deck. And I think the problem is that we're so used to thinking and doing happening at the same time. You know, you're doing the thinking while writing the paper. You're doing the thinking while putting together the slide deck. You're doing the design while writing the code. And the problem is, if you are delegating the doing, you have to pull out the thinking and do the thinking up front, and then have stage gates and so on, where you're making sure the thinking doesn't get lost, and that's a different, a different way of thinking. But that's a one of the common mistakes I see is people delegating that thinking, and then and also another mistake is then overvaluing and overemphasizing when it get when it appears to get things right, and underrepresenting the things it gets wrong. So there's a writing off of like, oh well it got the link wrong like in the example I gave earlier oh it got the link wrong but that's a minor issue and going oh but everything else it did really well and assuming that everything else it did really well rather than saying I've probably got the same error rate in everything else I just haven't applied the same level of rigor and so as a result when you then think that way you go okay how do I build this in such a way that I can actually check this, that I can make sure that it's valid at the end? Interestingly, I'm working on a follow-up book, and I've been one of the things that I learned from doing Accrual Intentions was that I was able to use AI to help me write the book because one of the first disciplines I did as part of the experiment was got it to create an observation log where we captured the lessons all the way through with dates, stamps, and so on, so that there was content, and therefore it wasn't having to hallucinate what happened.

Speaker 1 42:15
It was like it was it could access my lived experience. Now it didn't do this with the first book, but it has been doing it with the second book, where it'll go. Now be careful! I'm capturing these observations, but I know you know, and I created it where I could mark it as whether I've reviewed it or not, and it's basically going. But I notice you haven't reviewed them, and eventually I was like, you know what? I don't need to because I'm going to review the hell out of the book, and I'm going to review the hell out the structure and the example chapter and so on. So I know that there won't be anything that goes onto people's desks that I don't agree with that didn't happen, etc. Because ultimately, I am reading and reviewing and editing at individual word level for the output. Now, does that mean that if in a particular story it transposes a thing and changes the name and so on, does that matter? Well, it only matters if it makes it through to the end. But actually, I like the AI was even like saying, but you know, but how will you know? And it's like because I was there, I was in the meet the meetings that you are reviewing transcripts from and coming up with these observations and so on. I was in those meetings. I will know if you start attributing something to a different team member because I know that like I was there, and so that's a great example where actually I've worked out where my review points are, where I'm going to review really hard to make sure that the output the output is really good, and what I'm willing for it to then do in that process. And I think that is one of the failure point modes I see for people is they're not thinking how do I actually make sure it's any good. Even this morning, a COO of a business told me that, oh yeah, we're using AI all the time now. It's amazing. We built an entire forecast model with it in like a few hours. We've actually built our CRM using it now, and you know, without putting my head in my hands, I was kind of there going, "Okay, cool, that's good. How are you getting confidence that it's right? And they go, "Actually, oh yeah, that's a really good point. Actually, it does sometimes make errors, but you know, we're checking it, and it's that oh we're checking it, but it's we're checking it not this like we've got a robust testing process and automated tests and we're getting it to do it three different ways and then we're making checking those against each other like there's there isn't that robustness and I think that is one of the things that businesses are having to level up massively on is things that were good before, like you know, having well documented processes and all those such things, you you those were useful and important before. They're critical now. When you're you know when you're basically every for every human employee you've got, you've now got 11 AI team members that can do work. So you're massively increasing the amount. Output. Therefore, how are you making sure that your level of review and prioritization and clarity and all that kind of stuff levels up with that? And I think that's those are some of the biggest mistakes. I could go on. There's plenty of other mistakes to make. Those are good starting points.

Randy Johnston 45:15
Those are great starting points. In fact, this whole idea of thinking and very fixed review is critical to accountants for obvious reasons. And you know, I'm reflecting on the amount of AI slop that's out there. Some reports say 84% of all social media posts are AI generated, and I suspect it may be even higher than that. Further, not unlike you, Alexis, Brian, and I have a commitment where the things that we write, we're actually writing, and AI is not writing them. So, having produced monthly columns now in my case for 25 years for the publication, I just continue to write the articles and the insights. This idea to get it down on the half page, as you were saying, this that insights. That's where I think the humans really can play. Well, I know your time is valuable, Alexis. Brian, do you have other questions of Alexis? I suspect we could just keep listening and keep learning and learning, but other things that you want to know.

Brian F. Tankersley, CPA.CITP, CGMA 46:25
You know, the one thing I'm wondering about is, you know, we talk about one of the concepts that we talked about in a previous accounting technology lab is the use of tokens and how much compute power you're using. How you know? Did you go in and have to? Did you have to optimize things for what good enough is, you know, and so that the economics of of the token usage and the AI compute usage worked? What can you talk to us a little bit about that? You know, does it have to be perfect versus? You know, we make in sausage, we make in filet mignon. You know, can you kind of talk to us about that?

Speaker 1 47:03
Yeah, sure. So back when I first ran the experiment in February 2026, the economic model of the AI was even more bonkers than it is now. Which is to say that for $100 a month, I know that I was able to use many 1000s of dollars, possibly 10s of 1000s of dollars worth of AI cost. I won't say compute specifically. There's wider cost to it, but like I know that they were given, they were massively trying to incentivize people to use it, and are still doing so in an effort to both help organizations work out how do you actually apply this technology, and so they see value. But I also view as an element of and trap them, because once for those organizations that make the mistake of firing a load of people and placing them with AI, and then the AI cost goes up 1020, times, and you you don't have a quick option of switching back, so I think I was acutely aware of that back in February. Back then, I didn't have to do much in the way of optimization. That has changed over the following months within my own within my own real businesses and usage. I and I've had periods where the the the fiddling that's been done by the AI providers means that I'm running out of tokens every hour. Versus sometimes I've had weeks and I haven't, you know, I've used 50% and so on because they're clearly like messing with this and the model changes and some become more efficient and less efficient. All these sorts of things. However, what I have found is that it is definitely possible to be significantly more token efficient than most people are, and so I've run experiments where I've been able to improve token efficiency by 4,000x, and so as a result, I've kind of gone all right. And when I need to, I can totally then become more efficient. It's just that most of the time, I don't need to prioritize it, which should be a bit of a worry to the AI firms because, of course, there'll be an assumption of oh well, this usage is increasing and token usage and token usage, and all we need to do is then at some point change that pricing and suddenly you monetize all of that, and that doesn't allow the fact that suddenly you will massively incentivize people to become more efficient and so on. I've even run experiments where I've built set up a local model on my MacBook, where I built like I used AI to help me build a Chrome plugin that allows me to transcribe me to talk. It then transcribes it, which that's just a tool within like Chrome anyway, like you don't need it. You need need AI for that. But then it routes that into a local AI that I've got running on my computer, along with a mini prompt that I've provided. For example, write in the form of emails, or write in French, or in the form of a pirate. And then it's doing that, and none of that is going on to the internet. But I used AI to build me a thing that then costs me nothing over time, and I think that that's where I see a lot of this going. You know, I think that if I had to guess, I'd say that you know Apple may end up being real winner here, where you've got devices, you know, owns a load of the devices that we have that already have all of the components that you need for voice and visual and all this sort of stuff that actually most of the use cases of AI don't need the frontier model, and so that's where the value gets captured. But I think that that that experience that I had of going, oh yeah, this didn't cost me that much, but could have done, and then in subsequent months, looking at it and going, actually, but I can keep pulling this cost down over time has reassured me.

Speaker 1 50:48
That said, one other big mistake I would suggest people avoid is giving the keys away to big AI firms, and the way that I see that happening is that they will set up one of the most common is setting up projects within an AI tool, and then within that project folder, like workspace that you perhaps have then shared with other colleagues, you dump your files and so on. And every conversation you have in that project is also searchable and usable by the AI as part of that conversation, and so I see people going, and it's getting more clever, and it understands more about what we're doing, and so on. And there's two problems: one, that ignores context window problems, where actually you've got a load of stuff in there. Is it all relevant for the specific task in hand? You're not able to manage that context, so you can get worse results, or it just is expensive, and and the ability to make that efficient becomes very hard. The other problem is that at any point when you then go, actually, I want to use that other model because this one suddenly become more expensive, or they've moved all their data centers to the US, or you know a particular jurisdiction has meant I'm now not allowed to use it, or whatever. You all of your organizational knowledge and process and guidance and context is all sat in there, and getting that out then becomes hard. So instead, everything that I did was I made sure that my all that context and so on was on a separate system that the AI then points at using MCPs rather than is sat on their tooling, and I think that that's one way to protect yourself against the inevitable cost fluctuations and problems that we're going to get down the line.

Randy Johnston 52:35
Yeah, and see that makes delightful sense to me because the we know that the AI companies have been quote selling under cost, and you know how long that's going to last. So that's an issue, and also this whole idea of protecting your data and having the right, being able to move models and so forth, all big risks. So Alexis, any other parting thoughts from your side? Key things that you think listeners should know. Again, we're trying to leverage your experience of building these agents and being thoughtful about the impact on business. So, any other key things to know?

Speaker 1 53:15
Yeah. So, one of the things I'd say is that it's quite common if someone hears everything I've said is to go okay. They can kind of fall into a couple of camps. So one is they go okay. Well, it sounds like it's quite hard to get the value out of the technology. You know, you have to set up processes and guardrails and human checks and all this sort of stuff. I'll just wait until it's good enough. I think that's a mistake. I think that the technology is already good enough. It's just that you need all the guardrails and the context and so on, which you're going to need anyway. Like it doesn't automatically know about your business and what you want to do. And one way I bring this to life is to say, you know, when I'm talking on stages, but it applies to the audience. Every member of the audience listening right now, you've now got 11 plus AI team members that are ready to work for you, and imagine those were human team members. Imagine that I, out of the goodness of my heart, I'm going to loan you 11 AI team members, and they're all sat in the coffee shop just round the corner from you. Are they able to build your business, help you succeed, do anything for you as it currently stands? No, they need context. They need you to tell them what are you even trying to achieve for whom they need you to give them the guidance on what do you want each of them to do so that they're not all trying to save the whole same problem 11 different ways like that's true for human team members it's true for the AI and so regardless of how good the AI becomes you're going to need to give it that context and so you need to start now but then the opposite end of the spectrum I see is then people then say, "Oh, I need to do a massive transformation project. Then we're going to document every single process in the business. We're going to create every policy, every guardrail, etc. and do it as one big bang. That fails too, and it fails because it takes a long time for you to get the results. Also, you're you're not learning. Through that journey, because you're trying to do everything, you're trying to build it in this waterfall approach, building foundations. Instead, the best approach is to pick something in your business that is either incredibly frustrating and painful for you right now, or is a current missed opportunity for value to your client, and then work backwards from that to go. Okay, now that AI can do amazing stuff or whatever, let me spend proper time investing in trying to solve that problem. And the reason to start there rather than something easy is because you will find it harder than you expected. You will find that it makes mistakes and you have to build things and rebuild things and whatever, but that's fine when you're solving a meaty enough problem. It's not fine when you're going, "Oh, I'll just do this little quick win round the side, because then you quickly go, "Oh, it's the right pain, and it's not worth it. So you got to take a big problem, look, solve it, but particularly involve your team in doing that. Most team members will worry that the whole reason you're doing AI stuff is so that you can make them redundant, and actually, all of the experiments, all the research I've done is that the humans become more important, not less, and so you need to take them on that journey. So yeah, so I'd say that that's kind of some of my parting summary pieces. I can provide some suggestions on links and resources for people to learn more, and some of the things that I've highlighted that they might be able to use, and then also, I mean, there's a whole raft of stuff we could have gone to on my more recent experiments as well. But hopefully, that gives a useful guidance on where to start next.

Randy Johnston 56:36
Well, Alexa, such a delight to spend time with you today. We will put those links in the show notes. Thank you for the offer on that. And again, I can't think of a nicer way to spend the time. Your experience with the agents is really enlightening and helpful. And your and again, your guidance here superb. So thank you. And for all of you who have been with us again today, we look forward to seeing you again on another Accounting Technology Lab. Good day.

Speaker 1 57:06
Thanks very much.

Brian F. Tankersley, CPA.CITP, CGMA 57:07
Thank you for sharing your time with us. We'll be back next Saturday with a new episode of the Technology Lab from CPA Practice Advisor. Have a great week.

Transcribed by https://otter.ai