Better Stack · Transcript · · Updated
Microsoft Just Open-Sourced a Cheat Code for AI Agents (SkillOpt) #microsoft #ai #aiagents – Transcript
Video by Better Stack · Watch the original on YouTube
Summary
AI-generated from the transcript. Verify details against the source video.
- Microsoft Research open-sourced a tool named Skillopt that makes AI agents smarter by training a markdown file instead of retraining models or fine-tuning weights.
- During the training loop, the agent runs tasks to record rollouts, and an optimizer model reviews wins and failures to propose rule edits within a strict budget.
- Skillopt tested across seven target models, six benchmarks, and two execution harnesses, landing best or tied best in all 52 tested settings.
Use this video with Pro
Reading, copying and transcript downloads are free. Create saved AI results for this video with a signed-in Pro account ($9/month).
Transcript
Search ignores accents. All transcript text stays visible.
Microsoft Research just open sourced a tool called Skillopt that makes AI agents measurably smarter and it does this without retraining the model or fine-tuning weights or even touching the prompt by hand. What they trained instead was a markdown file. Let me explain. So here's what's happening. Right now, if you want an AI agent to perform better on a task, you basically got two options. So option one, fine-tune the model, but that is expensive and requires access to the weights and locks you into one model. Option
two is to hand write a better prompt, but that could be brittle, manual, and you're mostly guessing at that point. But Skillopt proposes a third path, treat the agent's skill document as the thing you actually train. And the way it does this looks almost exactly like a machine learning training loop. The agent runs a batch of tasks with its current skill file and records everything, every message, every tool call, every final score. And that's called a rollout. Then a separate optimizer model reads through the wins and the failures
and looks for reusable patterns, it can turn into concrete rules. And that is called the reflection step. And from there, the optimizer proposes edits to the skill file. So it can add a rule, it can delete a rule, replace one under a strict edit budget. And that budget acts like a learning rate, but for text, it stops the optimizer from overriding rules that already work, but leaves room for improvement for new ones. But here's what makes the whole thing actually hold together. No edit gets accepted just because an optimizer likes it. Every candidate has to
beat a held out validation set before it becomes the new skill and rejected edits get stored in a buffer. So the optimizer remembers what didn't work and stops going in the same bad direction twice. So does it actually work? Well, across seven target models, six benchmarks and two execution harnesses, codecs and cloud code skill opt landed best or tied best in all 52 settings they tested every single one of them. And what's really interesting is that the final skill file is
portable. So in this particular example, they did they took the best skill dot md file trained inside codecs and dropped it straight into cloud code with no extra training. And it still worked and picked up a 31.8 point gain on spreadsheet tasks. They also took a skill trained on a larger model and handed it to a smaller one. And that gained a 15 point boost without any retraining. So that proves that this skill isn't capturing quirks of a specific model. It's capturing how to actually solve the
task. So for years we've treated prompts as disposable texts you tweak by hand until something works, but skill opt treats them as trainable artifacts and gets results that compete with fine tuning without ever spinning up a GPU. So I think that's genuinely a very clever solution solved in a very efficient way. I'll put the links to the GitHub repo in the description below. If you want to stay up to date with the latest open source tools and frameworks, be sure to subscribe to the BetterStack channel.