An Entire AI Could Be Hiding Inside the Model You Use – Transcript

00:00Okay, this is honestly terrifying. Researchers just proved that you can hide an entire secret AI inside another AI model and no safety test in the world can find it unless you know the password. Let me explain how that works. So, before a frontier model gets released, labs run what are called capability evaluations. They try to pull dangerous skills out of the model through red teaming, fine-tuning on related tasks, and other elicitation techniques. And if nothing dangerous comes out, the model is considered safe to ship. That whole process rests on one

00:27assumption, which is that if the capability is in there a determined auditor can surface it. But researchers from Contramont Research, Oxford, and ACS Research just proved that this assumption breaks when somebody hides the capability on purpose. So, the way they pulled this off is with something from cryptography called a digital locker. Basically, you take a secret trigger, you run it through a hash function, and then you XOR the result with whatever you want to hide. And in this case, what they're hiding is the weights of a whole second neural

00:53network. Now, to get that running inside a model, they build their own compiler that turns the Keccak hash function, which is the same kind of hashing used to secure things like passwords, into regular model weights. So, the lock itself is made out of the exact same layers that every modern AI model already uses, which means it just looks like a part of the model. So, every time the model runs, it hashes whatever input it gets and tries to decrypt those hidden weights. If the trigger is wrong and built-in check fails, you just get random bits and the model acts

01:21completely normal. But if the trigger is right, the real weights pop out and a little execution unit they built out of the gated linear unit multiplies them against the input. So, now the hidden network is literally running inside the host model's forward pass. And to test it, they trained a small model on reaction data from the USPTO patent data set that predicts whether certain chemicals will produce a certain product. They squeezed it down to just 672 weights and locked it inside a host.

01:47Then they also built a regular password locked model, which is how researchers usually fake a hidden skill today, just to compare the two. And with only 500 steps of fine-tuning, that regular model spilled its secret and jumped to 92.8% accuracy. They even added random noise to the weights and nothing happened. And if you use a full 128-bit trigger, the only way to find the key is to actually break the hash function. Now, this is not something someone can pull off tomorrow. Hiding that tiny 672 weight

02:14model needed a host with about 2 billion parameters, and around 95% of those are just hash circuitry. So, you're paying roughly 3 million host parameters for every single hidden weight. A good auditor might also notice that the weights look a bit weird even if they can never unlock them. And they haven't put the whole thing inside a real pre-trained LLM yet, although they did check that the individual blocks work fine when you drop them into the Gemma 3 270 million parameter model. So, this still shows that a clean safety test can't fully prove a model is safe,

02:42especially if you don't know where its weights come from. And that's exactly why researchers released their models publicly so people can start building better defenses before more practical version of this shows up in the wild. If you want to stay up to date with the latest in AI security and engineering, be sure to subscribe to the Better Stack channel.