Mind The Prompt · Transcript · · Updated
Apple Put a 14GB AI on Your Mac. Here's What It Can Do. – Transcript
Video by Mind The Prompt · Watch the original on YouTube
Summary
AI-generated from the transcript. Verify details against the source video.
- Apple's third-generation AI models on the Mac, referred to as AFM3, include a small core model with 3 billion parameters and a big core advanced model with 20 billion parameters.
- Apple's on-device model remains heavy on the SSD by taking up to 14 gigabytes of storage while staying light in active memory.
- Apple's developer documents state that the on-device model may not handle basic maths, writing code, or logical reasoning.
Use this video with Pro
Reading, copying and transcript downloads are free. Create saved AI results for this video with a signed-in Pro account ($9/month).
Transcript
Search ignores accents. All transcript text stays visible.
Mac OS 27 put up to 14 gigs of Apple's own AI on my Mac. I found it in storage settings while clearing space for a different AI. It turns out there are two Apple models, and the Mac in this house with the most memory got the smaller one.
Which one yours got, and why 64 gigs didn't earn the big one, I'll come to. The AI I was making space for was GPT- OS 20B, a free 12-gig download. Apple's takes up more room than the one I was clearing room for.
And 14 is Apple's ceiling. MacRumors found 20 and a half gigs on an M4 Pro Mac Mini, and one owner reported 30. You can switch its features off, but the models stay on the disk, >> >> and Apple gives you no way to remove them.
In June, I'd read Apple's post about a 20-billion parameter model on the Mac itself, and thought, "A free 20B on every Mac?" Then I never typed a word to it. What's in the 14 gigs?
Since September, one of two models from Apple's third generation, which it calls AFM3. The small one, core, has 3 billion parameters. >> >> The big one, core advanced, has 20 billion, but uses only one to four billion of them for any single request.
The whole 20 billion lives on the SSD, and only the parts a request needs come into memory. Think of a jukebox. The cabinet holds every record, and only one or two are ever on the turntable. Most of what I'd read still called Apple's model a little 3-billion parameter thing. On a newer Mac, that's a year out of date.
And there's one kind of harmless request the small model turns down about half the time. I'd have typed one in my first five minutes. Which kind I'll save for near the end. First, why people like it. It's free, with no usage limit, and testers watching the network saw on-device requests never leave the Mac.
And it's quick. A benchmark posted this month on an M5 Pro had the big model writing 78 tokens a second with the first word after about 6/10 of a second. On an M3 Pro and an M4 Pro, testers measured 52 to 71.
A downloaded 8B Qwen 3 writes about 27 on a base M5. Different chips and single runs, so call it roughly two to three times quicker. A Reddit post simply telling Mac owners the command existed got nearly 900 upvotes, >> >> and one owner called it near instant. If it felt that way to you, it was.
What nobody quotes next to the speed is how much it can hold. That's the context window, the space for one whole conversation. On the small model, it's about 4,000 tokens. On the big one, developers measure about 8,000, though Apple's documents still say four.
The instructions, your text, and every answer it gives all share that space. Your request is the fixed part, the conversation grows because each reply stays in the window. At the usual 3/4 of a word per token, which is my assumption, the small one holds about 3,000 words and the big one about 6,000, questions and answers together.
It's a postcard, and the reply has to fit on the same card. One developer's prompt of just over 4,000 tokens was rejected by the small model, and one just under went through. In a chat, when the space runs out, you're told the transcript exceeded the model's context size, and that conversation is over.
Apple built this for a paragraph at a time, and at a paragraph, it's very good. Now the part I had backwards. I assumed 14 gigs on disk meant something heavy in memory. One developer measured Apple's model service on a 16 gig M3 Air at about 450 megabytes idle and 650 while answering.
>> >> GPT 0S, the download I was clearing space for, is a 12 gig file. A 16 gig M5 Air lets the graphic side use about 12.7. So, loading it leaves about 600 megabytes spare. That's roughly all the memory Apple's model needs while it's answering you.
>> >> Apple's model then is heavy on the SSD where you don't notice it and light in memory where you would, which is the opposite of the download. >> >> Nobody I found has measured the big model's memory on a Mac though and Apple publishes no figure.
Why does one owner call it near instant and another pretty useless? >> >> I count three causes. The first is which model you got and which macOS. Apple warns developers the model changes when you update and one tester's scam filter changed its answer on 11 of 227 messages.
The second is the shape of the job. The tidiest test I came across is on GitHub. A developer ran Apple's model and several small downloads, all 4B or under, on one M1 Max. 13 tasks of 40 to 100 items, every answer graded by a script.
>> >> Run twice, Apple's answers matched on 300 out of 300. Rewriting inside a word limit, Apple's model scored 85% against 55 for the best of the quick downloads. Summarizing with the facts intact, 100.
The third is timing. Leave it about 6 seconds and it unloads. So, the next answer takes 2 and 1/2 seconds to start instead of 4/10. Chatting, you won't notice. A script with gaps pays it on every call.
How does it compare with a download? >> >> Apple's own graders put last year's small model against Qwen 3 4B and about half the time couldn't pick between them. The rest split almost evenly. >> >> Prasad Kackar's test agrees. He gave 21 checked tasks, coding, maths and facts, to three models on a 16 gig M3 Air.
Qwen 3 4B got 20 right. Apple's model and an 8 billion parameter Llama got 18. That was last year's model on macOS 26. For the big new one, I found no run at all against a download on the same Mac.
The nearest is that scam filter, where Apple's big model scored 95% and GPT-20B 96 and 1/2 on a company servers rather than a Mac. And the gap is too small to call. On a quiz, it's a 4B keeping up with an 8B, >> >> and that isn't where it lets you down.
And Apple is unusually frank about where. >> >> Its developer documents say the on-device model may not handle basic maths, writing code, or logical reasoning. >> >> The maths example it gives is counting the bees in bagel.
There's one. I counted twice. Apple's description of last year's model says outright it isn't designed to be a chatbot for general world knowledge. Testers keep finding out why. Asked for help in terminal, it made up a homebrew command that doesn't exist.
And the new Siri chat isn't this model talking to you from inside your Mac. >> >> Apple's architecture post puts its world knowledge answers on Apple's servers. On a Mac, only the on-device option in shortcuts and the FM commands default model stay on the machine.
>> >> Now, which one did your Mac get? Apple's support page draws the line. An M3 or newer with at least 12 gigs of memory gets up to 14 gigs, >> >> which developers confirm is the big model with the 8,000 token window.
Every other Apple Intelligence Mac, M1s, M2s, and 8 gig Macs, gets up to 8 gigs and the 3B. The Mac with the most memory in this house is my wife's 2021 M1 Max with 64 gigs. It got the small one.
My own 32 gig laptop has a newer chip and got the big one. So does a 16 gig M5 MacBook Air. >> >> Memory didn't decide it. The chip's generation did. And Apple's only explanation is that the big model is unlocked by and optimized for its most capable chips.
To check yours, open the Apple menu, choose about this Mac, and read the chip and the memory. My wife took the news calmly. Her one question was whether she could have the 8 gigs back instead. And I had to tell her macOS 27 doesn't let anyone do that.
Now, that harmless request, >> >> it's a message that merely brushes a sensitive subject. In that M1 Max test, Apple's model refused 29 of 60 such messages. >> >> On the rerun, 30.