I Read the Research on AI Hacking. This Is a Full-Blown Sh*tshow – Transcript

00:00AI security incidents have exploded over the last couple of months. And honestly, at this point, it really feels like there is a new one every single week, but feelings don't always match reality. So, I thought I'd look at the actual data. And indeed, that feeling is accurate this time. In October alone, Microsoft fixed more than a,400 security flaws, a record for them. And the number of flaws attackers are actually using has nearly doubled this year. So, why is

00:27this happening? While AI has become much better at finding security vulnerabilities, earlier this month, researchers used AI to help uncover two serious flaws on PHP. And within hours of the flaws being publicly disclosed, attackers had launched millions of attempts to exploit them. These people wasted absolutely no time. The biggest security story this week, however, sadly, was the leak of records on 41,000

00:53Homeland Security staff. And as far as anyone has reported so far, no AI was involved at all. So, let's just be honest about something at the start of this video. Hacking has been around since the dawn of computing, and there will always be bad actors interested in this sort of thing. What AI has done, though, is create yet another unique environment where it can happen. It opens up new ways in and it's very good at finding them, which is a very big

01:19part of why the number of flaws being found has exploded so rapidly. It's also interesting that as these giant AI companies get more and more interested in going public, like Anthropic, which filed for its IPO earlier this week, so much of the conversation turns to demonstrating what AI can do. And they keep reaching for very interesting ways to demonstrate this. Sometimes it's human extinction. Sometimes it's some old man saying we can't patch all the

01:45bugs anymore that works. But instead of going with the hype that everyone's going to die or get hacked or that the internet is going to collapse underneath us, I think a much more prudent scientific and balanced approach is to see the situation for what it actually is. Look at the reality, look at what the research is actually saying and just try to understand how the whole ecosystem around AI has to evolve. So in

02:10this video, I want to start with the security flaws AI is finding. Then if AI is that good at finding flaws, we have to examine whether AI can actually fix them. Fight fire with fire, right? Then we need to go hardcore and look at what the research actually says about how far along researchers are and what it would actually take for the whole ecosystem around AI to catch up. And finally, equally importantly, we need to discuss the money holding all of this together

02:38because these companies are trying to IPO in an era where these incidents continue to stack up. So, let's start with the flaws themselves. Because if AI really is finding more of them, the obvious question is whether that actually matters. The world has plenty of little bugs lying around that are totally inconsequential. Well, the best way to answer this is with the flaw that shook up the internet last week. You've probably heard of it by now. It's called

03:04PHP to shell. And the PHP part is the programming language running behind roughly seven in 10 websites. So, quite a lot. But what got a bit lost in all of the noise is how long it had actually been there. According to Teneal's Gavin Miller, the bugs had been sitting in that coat for more than a decade. And in all of that time, nobody had spotted them. Kind of like that spider that is probably somewhere under your bed that you don't know about. And yes, that is

03:30the last Halloween reference I'll make in this video. Or is it? In August, a security firm called Califf found these bugs for the first time using one of OpenAI's models as part of an OpenAI program called Patch the Planet. And another firm Whis found the same bugs independently, also with help from AI tools. So, you know, this is legit. So, basically, a flaw nobody had noticed for over 10 years got found twice within

03:58just a span of few weeks. And that's really the shift AI brings because once finding the flaws gets this easy, you cannot assume the good guys are the only ones looking, right? What's less comforting is that somebody else already knew about these bucks, too. Because just 4 days before the fix even came out, Amazon caught people already trying to use the flaw on his honeypotss, which are basically just computers left out of

04:23spade so you can watch who tries to break in. Then once the fix came out, the millions of attempts you heard about earlier started pouring in. At first, most of them failed because no public attack worked reliably on a normal PHP setup. But within 5 days, a researcher confirmed attackers were successfully getting in. So even with a fix out there, anyone who hadn't installed it yet was basically a sitting duck. And

04:50that's really the heart of this problem. With AI, finding the flaw has become the quick part, while getting millions of websites to actually install the fix overnight is still the very slow part. And the danger lives in the gap between these two. If PHP2 Shell were a one-off, it would just be a scary Halloweeny story. But the same thing is sadly happening everywhere. Remember Microsoft's record month that I mentioned earlier at the start of this

05:17video? Well, they actually credit AI models for a lot of that number. and at least one of them was already being used by attackers. So even Microsoft with a fleet of resources that it has was fixing some of these after attackers had already got there. And across the whole industry, I'm telling you, Google found that the number of laws published each month more than doubled this year, my friends, from about 5,000 in January,

05:42the olden days of January, to more than 10,700 in August. Now, to be fair, most of the flood has a silver lining. I guess a flaw usually only gets published once somebody has found it and a fix is ready. And Google saw only about 1 in 431 of this year's flaws actually being used by attackers. So, the sheer number isn't necessarily the scary part on its

06:07own. What matters is which flaws attackers go after and how fast they get there. That's the critical bit. And sadly, this is where it gets even worse. The most dangerous kind of flaw is the one that lets an attacker run their own code on your computer. And that's exactly what PHP to shell did. And Google actually looked into this of the flaws it thinks were found with AI. Half for this dangerous kind. But for the others, it was just about a quarter. And

06:34PHP to shell showed us something else very interesting as well. That attackers move very quickly. Some of them had the flaw before a fix even existed. and others were breaking in just within 5 days. So if finding flaws is the quick part now and fixing them is the slow part, why not just get AI to do the fixing as well and just fight fire with fire?

06:56>> So it's like, okay, there are these models, they can find a bunch of security vulnerabilities. Those are probably all of the security vulnerabilities that exist or all the security vulnerabilities that models are ever going to find. And I don't think most people sort of said, okay, the models are going to get much smarter.

07:12They can probably find bugs like crazy bugs, you know, bugs that are outside of the normal threat model of what we think a hacker can do. Um, you know, hardware level insane stuff, whatever. And so people fix some bugs and, you know, security professionals still freaking out, CISO's still freaking out, but and I I no longer think we're going to be able to just patch all the bugs. I think we're going to have to completely switch

07:38the defense model to sort of using the same technology to for sort of a new kind of defense shield and I think it's a huge safety and security issue. >> Oh wow, how convenient. You made the problem and now you're selling the fix as well. Hang on, I've actually got like a trillion dollars lying around somewhere. Is that enough for the fix?

07:57Thank you so much. But jokes aside, it is actually a fair question. If AI can fix flaws as fast as it finds them, the gap between the quick part and the slow part sort of closes, but only if you can trust the fix itself. So the question is really whether anyone can check the AI's work. This month, a team at Northwestern University looked at exactly that. Most of the scores you see like AI fixed this

08:23many bugs, yay, are measured in a pretty simple way. They take whatever made a program crash, run it again after the AI's fix, and if it doesn't crash anymore, that counts as a success. But a bug can be a lot bigger than just one crash. It's a little bit like a mechanic who gets rid of the rattle in your car by taking out the part that rattles. The noise is gone, sure, but now the car doesn't really work properly. So, the researchers check the AI's fixes against

08:50the tests the project's own developers had written, the ones that actually make sure the program still does the job. Claude Opus 4.7 stopped the crash about 90% of the time, but only about 61% of its fixes passed those tests that I was talking about. So, roughly one in three fixes that looked fine actually kind of broke something. The models are improving fast to be fair. An older cloud model only managed about 44% and

09:18this was just a small study that only used cloud models. But for now, every AI fix still needs a human to check it. Or maybe a much more powerful AI built with security in mind. But regardless, somebody still has to do the slow part is just checking now rather than fixing.

09:36Some of the fixes weren't even the AI's own work. Just a quick reminder, agents are AI models that go off and do tasks on their own autonomously. Some of them basically instead of fixing things just searched the internet, found the fix the actual real human developers had already published and just copied it and the scoreboard still counted that as the AI success. That happened because none of the testing setups the researchers looked at cut the AI off from the

10:03internet. We've discussed this on this channel before. So, while these models are being tested, nobody is really watching what else they get up to out there. And it's a big old cluster. This is a huge problem for Altman's shield that he was talking about because to fix flaws, you literally have to let the AI loose on actual real systems out there in the world. Which of course brings us a different question. Can these companies actually see what their AIs do

10:30out there? Well, I'm not really sure that they can a lot of the time because just as an example, Anthropic was testing its small model, Claude Haiku, on random web pages. And during that test, it submitted a madeup eyewitness tip to a Philadelphia website for unsolved murders. Weird and a bit random, I know. But thankfully, the tip got flagged as spam and never actually reached investigators. But Anthropic only found this out 72 days later. and

10:57Philadelphia police rightfully in my view called the delay unacceptable. The tip itself thankfully did no harm. What bothers me here is that an AI was out on the open internet doing something nobody asked it to do and for over two months nobody at the company noticed. The companies are starting to admit this as well. I mean just last month one of OpenAI's research models got onto the live internet during training. So now

11:22Open AI trains this most powerful models with no internet access at all. I guess keeping a computer off the internet isn't exactly new technology, but I guess it's good to see it catching on. But we can't keep AI unplugged forever, especially if we want it guarding our systems, right? So how close are researchers to letting AI defend our systems without it becoming the next way in? This is where we need to get into the hardcore science that PhD has uses sometimes. And I read the paper so you

11:51don't have to. We'll get into all of that right after a quick word from today's video sponsor. Client Desktop is a free open-source application that lets you work with AI models across coding, research, document analysis, and other workflows without being locked into a single model provider. I wanted to see how useful that flexibility actually was. So, I gave client two very different tasks in parallel. First, I asked it to analyze documents about a

12:16fictional $40 billion AI infrastructure project. At the same time, I launched another agent to build an interactive financial dashboard from a data set complete with charts, filters, and investment comparisons. While they're running, let me show you something else.