AI在大企业中的应用才刚刚开始，人才供不应求？｜对话微软CoreAI副总裁Marco Casalaina – Transcript

00:00models by themselves don't do anything. They're literally 5,000 sensors on these cars. They spent like six months going all around BMW and gathering a data dictionary. And if this agent is impersonating me, it too can delete all of the files off of my computer if it really goes sideways. And Task Completion will be like, "You're good."

00:19But then I open the spreadsheet. I look at the spreadsheet. I'm like, "Uh, well, you know, the one and two bedrooms, those are pretty good, but those three bedrooms, man, they're way off." And you can't just say, "This price is wrong." You have to say this price is wrong and here's why. Because that why that's what goes into the rubric down the line.

00:35Trust but verify. So my job really is me and my team, we're the first people to try anything new at Microsoft generally, although also in the rest of the world as well.

00:49And so we really try these things and and determine how they're going to apply to our customers, how we can explain them to our customers because sometimes it's not super obvious what these new AI capabilities are going to be good for. Sometimes they are a solution looking for a problem, but sometimes they are a real solution to a problem and >> we really just need to kind of break them in and understand like how they work, the nuances of them and and that sort of thing. That's actually my like

01:16one of my most important questions and I think a question that audience care a lot about that is would just with the big models just eat everything because you try so many different new things right but the argument is you just need GPT you just need a codeex you just need anthropic model uh there's no applications on top of that what's your opinion on that >> well first of all let's bear in mind that models by themselves don't do anything I mean models by themselves

01:44they predict token by token but they themselves actually do nothing. So they have to be embedded within a harness of some sort >> for them to do anything. And so certainly our own M365 copilot is one harness. GitHub copilot is another. There are plenty of consumer harnesses that many people are familiar with.

02:01>> And so really it's the interaction of the harness and the model that produces the results that we see today. And these models are indeed as you say getting better and better. every day there's a new model from us or from one of the other model building houses that are out there and so they continue to improve uh for sure at the same time there is definitely a place for model customization one of the things that you're going to start to see now I think

02:28more often is models on the edge and in fact there is a model that I use on this laptop on which I'm talking to you quite frequently I use handy actually handy is an open source project for transcription it does transcription really really well. And so, you know, I wrote an email to you in advance of this podcast. I didn't type it. I probably dictated it. And I've gotten this dictation really dialed in, but it runs locally. This is a model that's running

02:55entirely locally on my laptop. Does not go to the cloud, nothing like that. Now, we're going to find more and more of these things as the hardware becomes more and more capable. I have a relatively new uh computer here, so it's pretty capable at at handling these models. But as the hardware becomes more capable of handling these models, I think you're going to start to see an explosion of models that have custom purposes that like like the transcription model. It's not a general purpose model. It doesn't do whatever it

03:22does transcription. >> I think the model can do it, but the other question is why do I need this one? uh when I can just have whisper for example or GPT uh real time to do this for me especially like you know clients are enterprise and I know it's actually a debate debate going on right now like so AI and you know enterprise shall we

03:48have our own model or shall we just give every data all the data to anthropic and open AI so like yeah what is the demand for this >> well I mean to be perfectly honest what the model that I'm using locally here is whisper. However, I have made a number of customizations to it. I mean, for example, let's say that I was commonly emailing you. I would probably need to teach it how to say Een, right? Like, it wouldn't just know how to say your name,

04:15>> which I may or may not be pronouncing correctly, but in any case, regardless of how I pronounce it, you know, will the model spell it correctly? >> I mean, maybe, maybe not, right? But if I adjust it a little bit, as I have been doing, then it's much more likely to to spell it correctly.

04:31>> So one argument is this customized model just works better for your scenario than out of the box. >> That's right. And you'll see that I mean we have seen that for many years with things like custom vision models. I mean for example in our own document intelligence. So in addition to the general purpose you know of course we have on Foundry we have with Microsoft we have many different models literally 11,000 like 500 models on the model catalog probably more than that now. uh and most of them are general purpose

04:58models like GPT and Claude and Mistral and stuff like that. But we also have this document intelligence suite which processes billions of documents a day and it is made specifically to read documents but moreover to find their layouts and stuff like that and that's something that general purpose models have a tendency not to do wonderfully.

05:18Like if you have a document and it has a table in it, that table cuts across pages. Then general purpose models have a tendency not to be able to stitch that together. >> But our document intelligence model can do that. In fact, it can do that so well that we also have a document translator.

05:35And the document translator will take the layout and reproduce it but in some other language. And if I translate from like English to Hebrew or Arabic, like a right to left language, it'll also flip the layout.

05:48>> Uh, it's pretty crazy actually. I mean, when you do it, it's it's kind of melts your mind a little bit when you you actually do these translations because it's like how did you do that? That's nuts. >> Yeah. And I think by describing the usage like the application of this model I realized something that is most of us like interact with new technologies because we are a consumer but Microsoft

06:13your job is mostly like talking with enterprise or large organizations and their use case can be very different. Yeah. So help us understand a little bit about their world like what are they doing how they are using AI. Well, actually this I can I can make a a very specific contrast between consumer technology and business technology. And let's not forget I mean Microsoft does have a pretty large consumer offering also, right? In terms of

06:39>> Xbox, Bing, Windows and stuff like that. So yes, we do have a consumer footprint as well. But >> but yes, it's true at least in my division that I am mostly talking to enterprises and governments. Although you know there are a lot of startups as well that use our technologies.

06:54Now, one of the things that I'm going to be talking about at the AI engineers conference in Shanghai is agents with their own identity. And this is an emerging trend. You're starting to see this everywhere. And you see this lately in some of the new consumer agents that have recently showed up on the scene.

07:09So, there's these consumer agents and when you log into them, the first thing they ask you is to give them a name. So, I I have one of these things. I named it Lily kind of randomly, actually. I don't know. I pulled that name right out of the air. So that kind of agent it has a limited form of identity. We at Microsoft we also have what we call autopilots and we announced that just back in May at or June I should say at our build conference. So we have this notion of autopilots. This is also an

07:35agent with his own identity but it means something very different for a business than it means for a consumer. So if I use my little lily agent over here, I mean ultimately when it takes an action, you know, if it goes and books travel or writes an email or any of that stuff, it's doing it as me.

07:53>> So it is impersonating me. So it may have a name, but beyond that, it's a pretty light identity. Like it doesn't really have its own identity for the purpose of taking actions. But one of the fundamental capabilities of our autopilots which also have their own identity. I mean yes you can give them a name and that's all well and good but they also get their own teams box their own email address their own file space and all of these kinds of things.

08:18They can take actions on their own behalf and crucially for businesses in particular and governments they run in their own security context. So when you have an agent that's impersonating you >> it is running in your security context.

08:32It inherits whatever permissions you have. So, for example, like if I have an agent impersonating me, I have the ability to delete all of the files off of my computer. And if this agent is impersonating me, it too can delete all of the files off of my computer if it really goes sideways.

08:48>> Mhm. >> But if I make an autopilot and it has its own identity, then first of all, it can't just delete my files. it can't even see my files because it's a totally different user just like you're a different user from me unless I specifically give it that access unless I affirmatively grant it that access.

09:05And so the way that we're using this notion of agent identities is really towards the principle of least privilege which you probably heard of that you know this idea that you want to give people and now agents the least number of privileges as possible for them to be able to do their job and no more than that so that they can't go sideways.

09:23that becomes especially important in this age of agents for the kind of government the governance that businesses and governments are looking for. That's interesting because I remember hearing acquire episode and actually talks about how Microsoft builds it advantage on top of the directory like is the directory that once like once you have the directory all the office

09:48uh software uh just inherits that very complex and behind the scene you know rights and permissions and files. Yeah. So that is actually playing in the advantage of Microsoft. But then the employees would be worried. Now I'm seeing an agent with identity doing what I can do. So like what does it mean for the workforce?

10:14>> Well, when you declare an autopilot, for example, when you create an instance of one of these things, it will report to you. So it actually reports to you. And the idea is that you can delegate some of your work to these autopilots. And there will be autopilots that our customers will build for specific purposes, certain types of customer support or partner support, those kinds of things. And then there will be more general purpose autopilots that we will use in the way like personal assistance and stuff like that that you use to manage your calendar and those kinds of