ATL272: The Gathering Storm: Open Source AI
Download MP3Welcome to the Accounting Technology Lab brought to you by CPA Practice Advisor with your host, Johnston and Brian Tankersley.
Randy Johnston:Welcome to the Accounting Technology Lab. I'm Randy Johnston with cohost Brian Tankersley, And both of us have been concerned about what is going to happen with cybersecurity risk based on AI. And this was probably illustrated even greater recently with the attack by a chat GPT on hugging face and the agents having the discussion among themselves. How are we going to do this attack? So it's like, what is going on here?
Randy Johnston:Now all of these large language models are improving. And, of course, regulatory environment here in The US is changing rapidly with the, you know, privacy security, AI regulation stuff being announced this week but kept private. So we do not even know all of the details as we're recording this episode for you. But bottom line here is we want you to start being proactive now on protecting your businesses. So, you know, in this case then, I'd like for you to recognize that AI, while being good in many ways, may be bad on cybersecurity.
Randy Johnston:Also, the day we're recording this was the day that Stanford announced that genetic modification and viruses have been created using AI. So it's been one of the concerns I've had for a long time, but that was officially announced in news articles earlier today. So notice there's lots of what I'd call the bad part of AI coming along. And so, Brian, I'm gonna turn it to you for just a minute to just kinda get your setup and thoughts on this as well.
Brian Tankersley:You know, Randy, I have to say that this you know, since Mythos came out, you know, since it announced we had issues with it back in April, I've been very concerned about this and the more research and the more reading I do, it seems like, you know, just like we had with where coding has changed forever, around the end of the year when Claude code became really kind of the dominant way that software got developed. You know, I I I think that security is going through a similar transition right now. And the real concern about this is that is not that you know, not about mythos specifically, but really about what's gonna happen when the open source models catch up with what's happened with the proprietary frontier models. Okay? So, you know, again, we we've seen I will tell you that I was absolutely shocked when you and I did our tech call for CPA America and was preparing for it.
Brian Tankersley:And I noticed that Microsoft had patched in, I guess, it was May or June, patched, like, 570 vulnerabilities that were ranked that were you know, it recognized vulnerabilities, CBEs, as well as about a half dozen zero days. And so it was it's one of those things where the volume of things that are being discovered by AI and the speed at which it's happening means that we're at a, I I think, a fundamental shift in how security has to work. And we really have to get away from the traditional, definitions based antiviruses and other tools to really proactive tools that are looking at, you know, get all the data out there. So it's a the landscape has really changed.
Randy Johnston:It has changed in a huge way. Now, Brian, this week, I actually was looking at an AI governance tool to control tokens access and so forth. It's a very fascinating tool. I know you were teaching, so you didn't have the chance to participate in the demo. But I am very concerned about the security of data from one AI platform to another, the securities of the model context protocols, the MCPs, and for that matter, vendors hoovering up and using client data, which could be a CPA licensure problem.
Randy Johnston:But, you know, it seems like a lot of professionals are looking the other way on that. But, you know, what I expect to have happen is a lot of the security tools that we've used traditionally will have more and more AI in them trying to take defensive positions for us. But, you know, it is clear that the numbers have already been shifting. The time to first attack after a vulnerability is exposed is well below an hour now, and it used to be measured in days, weeks, and months. Further, I think from our research, we've concluded that somewhere between 8090% of the attacks were executed by AI, not by people.
Randy Johnston:So, and I also believe in the AICPA town hall, they quoted a statistic, which I do not have the backing for. It's just what they said in the town hall, that cybercrime is the third largest business in the world today. So, you know, if those numbers are really true, oh my. And, you know, high value organizations are often targeted. Maybe that's ability to pay.
Randy Johnston:You know, we've said before, you know, follow the money. But, you know, humans are still among our weakest point on this with phishing and so forth. And, you know, I am getting numerous phishing attempts every day. I think my favorite one this week was the lawsuit filed against the company email. And, you know, it looked very valid, But, you know, looking at it, it's like, yeah, this is not a real deal.
Randy Johnston:We would have been served and, you know, blah blah. And and we have no threat of being served that I'm aware of. Okay. There might be, but, you know, it's I can see that if you were anxious or stressed or tired, you might just click through and you're compromised on that.
Brian Tankersley:Yeah. I will say it's been very interesting because I have c p a t e dot c h as a domain registered. And one of the things about having that is that I'm start because I'm on the domain contact for it, I'm starting to get more and more spam in German in addition to English. So, you know, it actually went through it went through my spam filter in pretty good detail for a session that I did yesterday as we're recording this, and it was amazing to me the extent to which we were being targeted. And, of course, the reason we're being targeted is because of this client confidentiality issues.
Brian Tankersley:You know, we have pretty much the dream identity theft set of data. And, you know, again, in many cases, we're also fiduciaries that are in charge of the disbursement of funds and have the controls over those. And so these firms are really the they're they're really the place where all of the different systems that are out there that are used to manage and control funds and to control organizations really fit together in those ERPs.
Randy Johnston:So you don't. I I don't quite understand that, Brian.
Brian Tankersley:No. What are you gonna do?
Randy Johnston:No. Oddly enough, another one of our associates and I were speaking yesterday, Steven Yoss, and he mentioned how many of the spam emails for him were addressed to Steve. And he doesn't go by Steve. Right? But he's getting emails addressed to Steve, and right away, he knows those have to be somewhat bogus.
Randy Johnston:And so Mhmm. You can teach your people for these types of tricks. But, of course, many of you are running client accounting services functions, or you have ACH information on your tax returns. So you've got wire authority and payment approvals, which, again, are, you know, very feature rich. And in a alternate an additional accounting technology lab, we've talked to you about updating your required written information security policy, discussing a few of these same concepts.
Randy Johnston:But, realistically, the credentials, the firm logins are keys to dozens of your client systems. And in many cases, you may even have credentials that are not for your systems but for clients' bank accounts, let's say, or other things like that. So because of retention policies, many of our businesses have documents on file for extended periods of time, and that also creates additional liability.
Brian Tankersley:Yeah. And I will say that again. I wanna kinda walk us through a one of the initial, one of the initial AI run campaigns in here. So this actually happened in last November, the state linked group, Pagenta coding tool turned it into an attack engine. They targeted technology, finance, chemicals, government had a small number of intrusions that were successful.
Brian Tankersley:Once they got in, they mapped the network, found sensitive systems, identified tested weaknesses, wrote exploits and code, harvested and validated credentials, moved laterally, and again, pulled the data out. And again, I think one of the things that's important to understand here is that when you're using AI, you can process all of this in parallel where, you know, when humans are doing this, they have to process this almost in a linear fashion. So they have to go from this to that to the other. And then when they're iterating on things, you can't, you know, you can't iterate on the other things right now because you're you have to really focus on doing the task at hand. And so, you know, on the other hand, with this, they could go out and and, again, the AI had perfect recall of the exploits that they had found that worked, and so it could try the ones that worked first and many other things like this.
Randy Johnston:So, Brian, you're talking about this. You know, in the past, we have talked about these attacks of bad actors. A classic one in this case was the Florida water supply. But, you know, as we're recording this session today, you know, the water treatment facilities have been attacked, we believe, by Russians at this point, and that has now spread into 12 different states. And I don't
Brian Tankersley:And we're seeing the we're seeing significant Iran involvement in that too because of the current Iranian conflict that's going on, as we record this in early August.
Randy Johnston:Yeah. And so notice where it originally was in Minnesota, you know, Minnesota reports over 30 municipal water facilities that have been attacked, but now it's in a dozen states, so I don't even have a good number on that type of thing. But, you know, I'm also quite cognizant as we're speaking, you know, that there's water shortages in Puerto Rico where people are having to hand carry water as we're talking about this. So just consider yourself for a minute. You know, electricity is valuable, but if you don't have any water, tell me how long you're gonna hang around.
Randy Johnston:Right?
Brian Tankersley:Yes. Sub one week. Yeah. It's Pretty often.
Randy Johnston:So, anyway but I don't wanna sound too over the top on this type of attack, but I do understand just how much of our infrastructure is at risk on this. And, you know, the attackers are using these tools that they can use alternate attack methodologies very quickly. And so fact of the matter is we are trying to have you think about how you stand up your defenses and how the attackers are trying to defeat your defenses or guardrails.
Brian Tankersley:Yeah. And so, you know, again, the the attackers jailbreaked the model. They posed as a firm running defensive test, and so then they went through and sliced things into innocent looking steps. And then it invented credentials and stolen data that was public. So, again, lots of so it hallucinated too.
Brian Tankersley:So that's all actually happening. But then we got to this spring, and Anthropics Mythos came out. And, again, when this came out, I think it's important to note that there was a row that happened between the US federal government, specifically the Department of Defense, and Anthropic about how the how the DOD was using Claude, and it it got pretty ugly pretty quick. The president himself even got involved in it. And, you know, I think that I think that looking at that now in retrospect three months later, it it strikes me that maybe that RAL had a lot more to do with use of Mythos as a hacking tool against other nation states.
Brian Tankersley:And so this preview was one of the largest cyber capability jumps ever happened, multistep offensive tasks in here, higher risk cyber requests that were, again, routing they were running through here. We had access runs going through the vendor so that abuse could be detected, planned, and reporting. And, again, Methos Preview went to went under project Glasswing so that they could come out with patches and boy, the patches have been forthcoming. I've noticed that we've literally patched thousands of, not thousands of different CVE in the last four months or so as a result of this AI detection and this AI hacking. And so I want you to I want you to understand that this capability right now is just in the frontier models.
Brian Tankersley:But I know, every all the experts we we read about are saying that that those capabilities are gonna be in the open source models, which the government can't regulate, by the way, in you know, within months. And so we're living truly in interesting times.
Randy Johnston:Yeah. And, Brian, as you look at this again, think about mythos, think about Fable, think about the current chat, GPT five six model, and so forth. All of these tools are so much more sophisticated than they were six months ago. And, you know, the case study that you just used was November of last year. So that was, you know, clear back in, like, chat GPT 53 or 54.
Randy Johnston:And, you know, you just look at how much more sophisticated these models are today. So, unfortunately, you know, I don't believe that governmental regulation is really gonna fix this. And, you know, in a prior podcast, we had discussed the, four leaders all trying to come up with regulatory models. And, you know, obviously, the Google's DeepMind CEO left the week that we're recording this podcast as well for, you know, different pastures, if you will. But this is not a solved problem.
Randy Johnston:It is way early in flight.
Brian Tankersley:Yeah. And I think one of the things that you've got to understand is that I think the cybersecurity environment for CPAs is going to get extra ugly this tax season because now we're gonna have I think, you know, by the middle of tax season, I think we're gonna have open weight models that are unregulatable that will will be used heavily for cybersecurity. And unfortunately, we have a lot of CPAs that choose to live with out of date technology. You know, they have already flawed home grade routers in their offices. They have they use unsupported operating systems without security updates.
Brian Tankersley:We have, you know, they have unsupported hardware. They have things that are past end of well past end of life. And so they have significant technical debt, technology technical debt, because they need to replace all this stuff. And unfortunately, it's kind of perfect storm because of the AI bubble that we're in right now. We you know, the price of things, as we've talked about in a previous episode, the price of RAM, the price of new hardware, the price of just about everything has gone crazy.
Brian Tankersley:Ubiquiti, that's one of the infrastructure tools that I use, you know, like Cisco or others. Ubiquiti actually now has a an a surcharge for RAM because RAM has gotten so expensive. So where it's, you know, up more than 400% since last year. So as we're again, as we're thinking about this now, we're really entering a scary place. And so, you know, if you haven't already done it, you really need to step up your cybersecurity posture now because we're going into a very bad neighborhood with very scary things going on.
Brian Tankersley:And, you know, again, if you're not really buttoned down, I think bad things are gonna be able to happen to you in this next year or so.
Randy Johnston:Yeah. And, Brian, even though you've talked about the cost of all the hardware going up, the cost of the tokens are cratering very quickly, including open source models and Chinese models. So talk about, again, Perfect Storm. We have this situation where the attacker's tools are actually dropping in cost very rapidly. So oh my.
Brian Tankersley:Yeah. And, again, the thing about this is that, you know, there are people, you know, just like we had with BitTorrent and with some of the peer to peer things twenty five years ago with music, there's nobody to subpoena. You know, there's no central person in charge of these open source tools. And once somebody posts something on Reddit, even if it's just out there for a few minutes for the weightings and the configurations and everything, you know, sure, you can shut it down on GitHub or Reddit. And then as soon as you do that, somebody's gonna post it somewhere else on Yandex or someplace where they can't shut it down.
Brian Tankersley:And so I want you to understand that when these things are out, they're out and there's no real coming back. So again, in 2025, the open model trailed the closed frontier by six to ten months. By mid twenty twenty six, the gaps four to seven months. GPT 5.2 matched the frontier model, GLM 5.2 matched the frontier model, four months older, deep seek matched a five month old thing. But again, as we're looking at this, you know, if this happened because mythos hit in April of this year, the first of the year is really going to be that eight month mark where things, where we think things are going to get crazy.
Brian Tankersley:And unfortunately your firms aren't going to be in a position to do anything to do significant things because you're gonna be so busy making hay, trying to take care of all your clients during that time of period that you where you don't have time to sell.
Randy Johnston:Mhmm. Okay. We talked about in the wisp podcast, think about how you're gonna respond if you're attacked, an incident response plan. And just note that we think the coming few months could be the time frame it happens in. We are gonna I'm gonna go a little bit further, Brian.
Randy Johnston:We did not talk about that in advance, but it's my opinion we may see a few of these attempts made during tax extension season this year, kind of quietly testing the tools, much like we are seeing some quiet testing of tools in the election seasons right now.
Brian Tankersley:Yeah. And I wouldn't put it past a lot of foreign actors and domestic actors too to do all kinds of crazy things around the election.
Randy Johnston:Yeah. Well, you know, we've had the social media election, and now we're gonna have the AI election.
Brian Tankersley:Yeah. It's pretty nuts here. So just kind of looking at this now, we have you know, again, just to kinda look at things. You know, we got a list of things that are over hyped in the left here. And, again, we do want humans still in the loop, and there are a lot of vendor claims in here.
Brian Tankersley:What's real, though, is that we have, again, like Randy mentioned, we have cheaper attacks. We have more targets. We have the machine speed shrinking the response time. Same tool. No breaks.
Brian Tankersley:And, again, there's no dispute. K? Everybody agrees that this is happening. Everybody agrees that this went on. I mean, the according to some of the reports I read, you know, of course, the government's not gonna comment on the record about this.
Brian Tankersley:According But to some of the reports I've read, Mythos actually defeated not only every browser and literally had hundreds of vulnerabilities in JavaScript and other web languages. It also defeated all the secure systems of the National Security Agency, which is one of the most secure button down environments in the world. So, you know, it it to for it to defeat all of those systems is pretty epic.
Randy Johnston:Yeah. And, you know, the models are still hallucinating, but one thing we don't want you to think, you know, it's not Terminator and Skynet at this point. I'll be back. But, you know, bottom line here is it is much more risky than you might see. You know, a lot of us go along content in our own little world, and we don't think about it until it affects us.
Randy Johnston:And I don't know which one of you might be the Us.
Brian Tankersley:Yeah. You know, the hard part about security, of course, is that there's really a really bad feedback loop. Many times you don't even know that something bad's happened. So, you know, we've listed out a few things here to do, you know, harden and rehearse things. You're gonna want to, if you're watching, you're gonna see higher attack levels and not higher attack pace.
Brian Tankersley:I will tell you that I have besides my environment where I've got my router that's connected to two different Internet connections, I've also got some public servers that I have root access to, and I monitor those. And I will say that the attack volume is going up significantly and there are a lot of things happening. So you're going to need to you're going to want to rehearse your incident response in here and again, to also shorten your detection to containment times now. So get better at fixing problems because it's very likely that you're gonna have those coming up.
Randy Johnston:And you may need to do a restore from backup. So as fundamental as that is, get the fundamentals right, including testing the backups. And beyond that, you wanna make sure you're looking at all your own AI. In other podcasts, we have talked to you about a three tier model, the productivity, the AI built in, the agents and MCPs that are building getting built. But I want you to take the time to inventory all the tools in your stack, Consider which data is actually leaving your firm and how, and check the audit trails now so you can also see what they look like when they've been compromised and make sure that you control your data.
Randy Johnston:Many of the vendors are trying to lock up your data, but, you know, you've gotta be taking the actions now because when security's working, you don't really notice that it's working. That's another one of the problems with security. It's when security fails that you now know that you've got a problem. So it's one of those things that's very hard to spend money on and feel good about it because it's like, yeah, threw this money, but I can't see it's doing anything for me. Yeah.
Randy Johnston:That is there is a bit of a FUD factor on some of this. But, you know, if you're waiting for things to be certain, that ain't gonna happen. It is just going to you are likely to be one of the ones that are attacked along the way here. So
Brian Tankersley:You know, again, the thing here is we're looking at this is that, we're not trying to be you know, I know we've covered a number of difficult topics this, you know, in the last month or so. You know, we had our session on GLBA and WISPs and other things like that, and now we're talking about this. And we really hope that we're wrong about this concern on security. Okay? But all the other people that I talk to and then I read that are that are commenting and everything, I I think everybody kind of has everybody's kind of waiting for the other shoe to drop.
Brian Tankersley:Okay? Because we've seen what's happened. We've seen the effect that can be generated with mythos and other things like this. And it's just, it's, you know, some of these problems look pretty scary. So that's, we feel it's our job to tell you what's going on.
Brian Tankersley:And sometimes we're giving you bad news, and I'm sorry.
Randy Johnston:Yeah. In fact, as you were saying that, a friend and associate Dave Bergstein has many times walked up and says, Rena, I'd love to listen to you, but you're always negative. And it's and I usually say, look. It's my job to tell you the good and the bad. And in this particular case, yes, Brian is absolutely right.
Randy Johnston:There's some stuff that I would consider negative here, but he knows me well enough. I'm an upbeat guy. I always may be a little too on a lot of this. So just understand that Pollyanna Randy is suggesting that you may well have some really ugly conditions in front of you, and I'm trying to keep you out of the storm. So that said, a delight to be with you again in today's Accounting Technology Lab, and we look forward to having you listen in again in the near future.
Randy Johnston:Good day.
Brian Tankersley:Thank you for sharing your time with us. We'll be back next Saturday with a new episode of The Technology Lab from CPA Practice Advisor. Have a great week.
Creators and Guests
