9. Introduction to Gen AI – Transcript

00:15Hello everyone. Today I going to give a very very brief introduction to uh the generative AI. I think there are a lot of questions that people have in their minds as to what is this new technology? Why is it why is it so powerful and things like that? I mean I'm not going to give you the sales pitch, right? So I mean must have heard all these kinds of stories AI generative AI models that are able to cure cancer and so on so forth, right? So but in reality what you need

00:42to think about is that it is not as if AI magically went and cured this dog's cancer, right? So what really happened was uh you know the AI acted as an always available guide you know something that was helping the humans look in the right direction and perform the right experiments and so on so forth. cut down significantly the time that is required, right? Cut down significantly the time that is required

01:08to uh uh do the research uh to find the drug, right? But still people had to go and actually you know synthesize, figure out ways to make the drug work and all do all the laboratory testing and so on so forth. So the whole domain process still needed to be uh you know completed. You still needed the experts who had the capacity to do that. So, and so it's not as if AI is a magic bullet that helps you uh cure cancer all by itself, but it is something that you have to work with. I mean, this is some

01:36this is a message that you will see repeatedly uh uh coming at you uh as scientists, as researchers, right? So, you are going to face situations where people are going to tell you that AI is going to do your job for you. You don't have to do research anymore. But that's not true. AI will help you do your research, right? help you create impact, help you you know find new things at a pace that was not you know thinkable

02:02right uh uh uh uh a few years back right so that is the state that we are in but there's a lot of work that needs to be done so it's good that you have some rough sense of what is this technology what is the you know the math behind it and so on so forth so I'm going to give this to you at a very high level right uh so there have been lot of major AI breakthroughs uh uh these are actually breakthroughs that were mediated by AI Right. And [snorts] there's a lot of hype that comes around saying that so so much billions of money being invested in

02:29AI and so on so forth. And there's also all this uh uh you know stories about how AI uh uh uh uh can take over uh uh this security threats and things like that and how people have to stop using AI and so on so forth. A lot of top personalities have been talking about this etc. But what is unquestionable is that uh AI is causing a big change in how research and how all scientific endeavors are being done. Now it's not

02:55just question of helping you write your paper, right? Uh but also a question of actually doing something more uh deep thinking along with you right. Uh so one thing to think about right is that uh chat GPT that everybody talks about and all the generative models that we have today was not really the start of AI right. So uh like this is the uh tweet from Sam Alman announcing the release of CH GPT. So it's like basically November uh uh end like November 30th right and

03:23it's not really the sort of AI has been around for a much much longer time. So just a couple of slides to give you some historic context and also to tell you more about uh uh situating uh the current uh wave of AI right so it goes back to Alan Turing who asked this question in 1950 right uh can machines think and then he came up with an answer which we all today know as Turing test which is if a machine can imitate a human so perfectly that a human uh you

03:50know interacting with a machine and another human is not able to tell them apart right so basically it was a question of uh behavioral testing right now. He didn't get go into the whole process of what is intelligence, what is thinking and all that. So if a human you know interacting with these machine is not able to tell it apart from another human being then you would say the machine is intelligent. So that is where it goes and the machines today have uh you [snorts] know can pass this very

04:17early version of the Turing test uh uh very easily I mean unquestionably right so uh so it's um it doesn't mean that they have become truly intelligent it just means that we have to relook at our uh uh uh tests right people keep talking about humanity's last exam the the the reasoning uh uh benchmarks and so on so forth people are getting pushing the boundary more and more in terms of uh what these machines should be looking at right and the term AI itself right this

04:43was coined in 1956 at the Dartmouth uh summer conference Dartmouth conference as it's called now right and this was convened by John McCarthy uh Minsky and Shannon and Rochester and they basically had a a group of about 10 12 very very top brilliant minds at that time get together and think about how to uh uh uh simulate intelligence right and so That's basically where they created the

05:11term AI and since then we have had a lot of stories come out, right? So and and and even real AI success is not new, right? So that's um Arthur Samuel in the picture and he's playing checkers on live television against a machine that learned to play checkers from scratch.

05:28Not a machine that was programmed to play checkers, but the machine that was programmed to learn to play checkers. This is one of the earliest uh demonstration of um an AI learning something non-trivial and this was done by experience right just by playing the game again and again right so that's how it was learned and in fact this early instance of what is called reinforcement learning right and of course there are also uh later models like Eliza which

05:53was more of a conversation agent that you could talk about your mental problems and your uh uh uh health problems and so on so forth and came out in the 60s and it's actually sold in sold in Radio Shack, right? So, it's like like amazing levels of prevalence back then even, right? It's not now, right? Of course, these these models, these games and things like they had their own limitation and then people quite kind of figured out that uh we

06:19needed newer technology to go farther, right? But that's just not it, right? Not just AI success, but AI hype is also not new. Uh [snorts] and so here is a headline from a newspaper. The Navy revealed the embryo of an electronic computer today that it expects will be able to walk, talk, see, write, reproduce itself, and be conscious of its existence. [snorts]

06:43And this headline came out from 1958, right? Barely a few years after uh Turing asked the question and the word the phrase AI was coined even, right? So there's always been very high elevated expect expectations about AI and it's always been going through this kind of hype and bust cycle and so on so forth.

07:04So here is a quick overview of that. So in the 50s and 60s the term AI was coined and the first AI wave happened in the next decade right or so and in about 69 the first AI winter set in right because people were showing that Minsky himself showed that uh the the the currently popular wave of uh uh neural networks back then uh could not solve very simple problems right which humans will have no difficulty solving and they kind of then discovered all the scaling

07:31issues that were there with these AI models it kind of went into like a winter right and Then the second wave of AI started uh uh you know around 83 when Hinton and his colleagues uh you know came up with back propagation that allowed you to train more more complex neural networks and then people who are working on other aspects of AI which did not involve neural network they also started getting a lot of attention and then some of the big success stories in that time period right so was like this

07:58world chess champion uh uh chess engine right so which beat Gary Caspro in match match play and Caspro made statements like I could feel human level intelligence across the room and that caused a lot of uh uh you know people to really think that AI has arrived here and so on so forth and uh so it was very hot for a couple of decades back then right so that's kind of when I was getting into my doing my

08:24PhD and things like that so I so there's a lot of interest and activity in the AI arena but then around 2000 people kind of figured out that hm all these promise of AI high cracking intelligence and getting into the market and solving really large problems and that was not happening right so these models were very brittle lot of fine-tuning handholding needed to be done and so people did not really have a a scalable mechanism for doing this so

08:50it went into uh what we call as a second winter right investments went down people in the uh in the investment circle were saying that hey look if you're saying that you are going to do AI or robotics then that's like the kiss of death for your company because nobody's going to invest in that Right?

09:05It went it went into that kind of a bad situation. And then of course with all these deep learning revolution that came about, right? And this was in the early 2010s, right? And then uh so you had a lot more successes in you know in computer vision, in vision, in language and in in speech understanding and so on so forth. And so all these uh you know your alexexas and your Google homes and

09:32all Google maps and search engines and all of these things start started having more and more AI embedded in them right and also this was the beginning of the first uh know AI companies right so I I call them AI first companies as opposed to uh uh companies that used AI to build a primary product right uh so companies like deep mind and open AI started forming And uh in in uh 2022 kind of

09:58marks the beginning right just slightly before that there were things like stable diffusion midjourney and so on so forth this kind of marks the beginning where people started delivering AI as a product you know a raw AI interface not not an AI that was designed to do a specific task right but some AI that you could work with and get it to do what you wanted it to do right not not in terms of just uh uh being a conversation

10:25agent or being something that helps you find the right products on the market and things like that, but you could just talk to it and make it do anything you want. So that is the birth of uh the current wave of AI. It's kind of we never went into a winter in some sense after the deep learning wave. We just became really hot summer, right? And then of course uh um know a few years back we had two Nobel prizes awarded to basically to AI discovery, right? So one was on the fundamental research in AI

10:53that's basically uh the physics Nobel went to Hinton and Hopfield and also the uh the other one the chemistry novel went to AIdriven discovery that of protein uh uh folding right protein structure prediction using AI right so this this hype no just keeping on going there seems to be no end in sight for it and today we hear about uh GPT6 Astra which is supposed to be very close to human level reasoning and all that I mean every time one of these models

11:20comes So that's the story you hear and afterward this yeah yeah not quite as good as we thought it is but still it's a significant improvement over the state-of-the-art and it's almost always right. So we are pushing the state-of-the-art back every time and it's it's it's it's a challenge right.

11:37So one thing to keep in mind as we keep discussing about AI is that AI is not a single technology right so so things like uh the models like Eliza and so on so forth where essentially rule-based systems right we call them expert systems right which could kind of the human designers talk to experts distill their knowledge into set of rules and then these AI models use these rules to come up with decisions right and then you know success stories like the chess

12:04engine uh typically used a lot of what we search, right? Enumerate all possible outcome and try to uh go to uh what what it thinks is the the best possible direction in which to expand the game, right? So that's basically the idea behind sets. Of course, it's not just used for gameplay. It's used for a whole bunch of other technologies, but that's the core of it. And then there are a set of things that based on logic and reasoning kind of combines the search idea and the rulebased system, but

12:30thinking about it in a more abstract uh uh representation. So that is uh uh that's one example of that was the Watson and uh Jeopardy engine from IBM which learned to play the general quiz game of Jeopardy very well right and then of course we moved into more of these deep learning computer vision uh uh like uh applications which uh used a lot of data uh to train a model uh with millions of parameters to do tasks that