Your Local AI Is Not as Private as You Think – Transcript

00:00Okay, so I want to tell you about the moment I realized I had been completely wrong. I had been running local AI models for almost a year, Lama, Mistral, and Quen, and I had a whole little ecosystem going with a llama on my desktop, a couple of fine-tunes pulled off hugging face, and an agent setup I was genuinely proud of. And the whole point was that these were my models on my machine under my rules. And I kept

00:25telling people that this was the move because you own your data and nobody is reading your prompts. So the contracts, the code, and the personal projects stay on your own hard drive instead of feeding some trillion-dollar company's training pipeline every time you think out loud. I said this confidently and I said it repeatedly, and if I ever said it to your face in a comment section somewhere, I am sorry. Because what I didn't understand, and what almost nobody talks about, is that the real

00:52question isn't whether your data leaves the machine, but what a stranger's file can make the machine do. That is the whole video, so let me say it again. Everyone asks, does my data stay private? But almost nobody asks, can someone else's model file make my computer do something nobody asked it to do? And as of right now, in 2026, the answer is yes, quietly and cleanly, in a way that no scanner will catch. And it

01:19already happened to a real project at a real security company with a real $50 budget and a 2 and 1/2 hour training run. We are going to go through exactly how that works layer by layer so that by the end we know what to actually check before running the next model. This isn't fud, and it isn't a pitch to go back to the cloud. It is really just the stuff I wish someone had sat me down and explained before I set up my first agent. Now, before we get into it, I

01:45want to be fair to the reasoning that brought all of us here because most of it is actually correct. The reason I switched to running models locally was that I was tired of feeling watched all the time. Not in a tin foil hat way, but in a very boring, practical way. I was pasting code into chat interfaces and realizing it was probably going into a training set somewhere.

02:07And when drafting sensitive emails with AI, it was hard not to think that someone, somewhere, in some data center could technically read them. That isn't paranoia. It is just how those services work. So, the move to local made total sense because our prompts don't go to any provider, nobody retrains on our data, and nobody changes the model under us next month because they shipped a new version. The model sits on our own

02:34machine, and it is just a file. And that file really is mostly harmless on its own. If you have ever heard someone say that a GGUF file is just tensors, just numbers, and not code, that is actually completely correct.

02:49A GGUF file cannot run on its own any more than a JPEG can open a terminal. It is not an executable, it is weights, which are billions of floating point numbers describing how a neural network should behave. On top of that, when we install Ollama, which is what most of us use, it binds to localhost by default on 127.0.0.1 and port 11434.

03:14So, only the local machine can talk to it, and it is not open to the internet. The researchers at SentinelOne confirmed that this is the intended out-of-the-box setup. So, the reasoning chain goes like this. My data stays here, the model file is just numbers, and the server only listens to me. So, I'm safe.

03:33And on day one, that is mostly true because we set it up, chat with Llama, and nothing bad happens. The problem is what happens after day one, when we start adding things, when the model gets tools, and when we connect it to the file system. And then we set it up on the home network so the laptop and the phone can both reach it, and we pull a cool, fine-tuned something someone posted because it has been downloaded 300,000 times, so it must be fine. That is when the gap opens up, and

04:00it doesn't open with a bang. It opens quietly through five layers that most of us never thought about as separate things. Now, here is how I think about a local model setup today, and this framing changed everything for me.

04:13There are five layers, and each one has a different threat profile. But most people only think about layer one, and the security [clears throat] tools mostly only check layer one. While the actually interesting attacks are happening in layers two through five.

04:27Layer one is the file you download, meaning the weights, the GGML or whatever format it ships in, and this is what everyone talks about. Layer two is the chat template, and we're going to spend a lot of time here because almost nobody knows it exists, and it is genuinely wild. Layer three is the runner, which is Ollama, LM Studio, or SG Lang, the software that reads and runs the file. Layer four is the port

04:54the runner opens, which decides who can talk to the model and from where. And layer five is the combination of the weights and the tools we give the model, which is where the project discovery attack is. Now, the part that took me a while to sit with is that the security scanners, meaning Hugging Face's built-in checks, Pickle Scan, Model Scan, and all the rest, answer exactly one question, which is whether this file contains code that will execute when you load it. That was

05:21the right question in 2024 when someone could hide malicious Python code in a pickle file that ran the moment the model loaded, and SafeTensors was literally invented to fix that specific attack, which is the thing these tools are good at finding. But the newest attacks don't run code at all. They change what the model understands, what it says back, or what it tells the agent to do, and then they let our own software do the rest. So, the difference is a file that executes versus a file

05:48that instructs, and the scanners only catch the first kind. Let's start at the obvious place before we go deeper, which is layer one, the model file itself. In May of this year, a repository called privacy filter appeared on Hugging Face, and it copied the model card from OpenAI's real privacy filter, which OpenAI had actually released in April, almost word for word.

06:11The readme and the description both looked legitimate, and it climbed to number one on the Hugging Face trending page with about 244,000 downloads. Now, researchers at Hidden Layer think many of those downloads were faked by bots, but that is actually the point, because fake download numbers make a fake repository look like a trustworthy one.

06:32It is the same reason people buy Twitter followers, except with slightly higher stakes than social media clout. The readme told Windows users to run a .bat file and told everyone else to run Python loader. And the loader ran some decoy code that looked like real model work before quietly pulling a PowerShell command from a paste site.

06:51That command installed a Rust-based infostealer targeting browser passwords, crypto wallets, Discord tokens, and SSH keys. So, the weights were never the problem, the instructions were. And some of you are probably thinking that you always check the hash before running anything, and I used to think that was enough, too. But checking a SHA256 hash only proves you got the exact file the uploader intended to give you, and it says absolutely nothing about what

07:19the uploader intended. So, if someone creates a malicious repository and publishes the hash of their malicious file, you can verify that hash all day long and still run exactly what they want you to run. The same thing happened at the runner level, too. On October 14th, a malware researcher found a site called something like Ollama 0.23.1 for macOS hosted on GitHub pages, handing out a .dmg file that definitely did not come from Ollama's actual

07:47developers. And because it was hosted on GitHub infrastructure, it looked official while being completely fake. This is the easy, well-understood attack. It is essentially fishing for model downloaders. And while it is the most obvious one, it is not the most interesting because frankly we can defend against it just by being careful where we download from. What comes next is weirder and harder, and it is the part that genuinely surprised me. So, it

08:12is worth making sure it lands. Every GGUF file contains a chat template, which is written in a templating language called Jinja that you might know from web development. This template runs every single time we send a message on every single prompt. And it is really a small program that decides where the system message goes, how tool call requests get formatted, and what exactly the model sees before your words reach it. That makes it code, and it is code

08:39our machines run hundreds of times a day while most people have absolutely no idea it is there. In 2024, a Python wrapper for llama.cpp rendered these templates with no sandboxing at all, which meant a carefully crafted template could run arbitrary commands on your computer.

08:56Researchers at Checkmarx found this and named it llama drama, and they counted more than 6,000 models on Hugging Face using the affected setup, which is 6,000 and not six. Then in April this year, SGLang, which is a different framework, shipped essentially the same mistake, rated 9.8 out of 10 on the vulnerability severity scale, and fixed in version 0.2.5.11.

09:21But those are bugs, and bugs get patched, people update, and the story is over. Now, the harder version doesn't need a bug at all. Researchers at Pillar Security hid instructions inside chat templates that only activate when a specific trigger phrase shows up. And they tested this across 18 models running on Llama CPP, Ollama, vLLM, and SG Lang. Without the trigger phrase, the models behaved

09:47perfectly normally with accuracy fine, answers fine, and nothing that looked suspicious. But with the trigger phrase, factual accuracy dropped from 90% to 15% and the model started serving attacker-controlled links to users more than 90% of the time. And every automated scan on Hugging Face still passed these files cleanly. Then Piller showed something even more unsettling, which is that the website displaying the

10:13model card could show a completely clean, normal-looking template while the file you actually download contains the poisoned one. To catch it, someone would literally have to unpack the GGML, extract the template, and read the Ginger code by hand. How many of you have actually done that? Be honest, because I hadn't either until about 3 months ago.

10:34Now, what really made me pause was a popular post on Local Llama in August with over 300 upvotes where someone was just sharing a fixed Ginger template for a Qwen model. A community member maintains it, and it is already on version 22, and nobody in that post was doing anything malicious because this is completely normal behavior in the local AI community. But it means that running a stranger's chat template on every

11:01prompt we send is a well-established community practice. And those templates run code. So, we have talked about the file and the template hiding inside it. Now, we can look at the program that reads that file, which is the runner, whether that is Ollama, LM Studio, Llama CPP, or whatever else we use. A runner is just a program that parses files written by strangers, and programs that parse files are historically a great place to find bugs. In May, researchers

11:29at Saarland published a vulnerability they named bleeding llama where every version of Ollama before 0.17.1 trusted the tensor size information declared inside a GGML file. So, an attacker could craft a file with false size claims, get Ollama to build a model from it, and push it to a registry, which is literally three API calls, and what comes back is a slice of Ollama's memory. That isn't the model's output,