Ansh Lamba · Transcript · · Updated
RAG Tutorial For Data Engineers (From ZERO To PRO) – Transcript
Video by Ansh Lamba · Watch the original on YouTube
Summary
AI-generated from the transcript. Verify details against the source video.
- Rag combines document retrieval with generative models to answer queries using external knowledge.
- Semantic search applies cosine similarity between the query vector and vectors in the database to retrieve the top similar chunks.
- Ansh Lamba uses Chroma as an in-memory vector store to store chunks and their embeddings.
Use this video with Pro
Reading, copying and transcript downloads are free. Create saved AI results for this video with a signed-in Pro account ($9/month).
Transcript
Search ignores accents. All transcript text stays visible.
In this new AI era, AI engineers will be in demand if and only if they know this skill. And I'm talking about DAG, retrieval, augmented generation. This is the only skill in this newi era which is aligned with the databases and data modeling. And nowadays, companies want data engineers to build these rack solutions just like other data warehouses. So it's the best time for
you to shine. That is why I have created this 3 hours long rag master class so that you can become a pro rag developer from scratch and this course will start with the fundamentals and progressively you will be able to write code to build the realw world rack solutions. Along with this we will also integrate open AI hosted models into our rag applications
and also load lms locally using docker model runner. The best part this course is full of cool visuals so that you can easily understand those complex topics plus all the notes and code examples are provided with this particular YouTube video. In short, this is the only video you need to master rack as a data engineer this year and make yourself AI
proof and just crack your dream job offer this year. So hit the subscribe button right now so that you will be able to get all the important master classes on this channel and let's get started with this amazing master class on RA. So what's up? What's up? What's up my data fam? How is everyone? You should be super excited today because
today we're going to talk about rag which is the most important area of AI for data engineers. If you ask me hey Anlamba we have AI everywhere everywhere everywhere. Okay everyone is talking about AI for data engineers what is the most important thing in AI to learn? The answer is rag rag rag rag. And that is why >> [snorts]
>> Just go to LinkedIn, just go to any job portal, okay? And just search for data engineering roles or let's say AI data engineering roles and you will see rag everywhere. And do you know what rag is for data engineers? Let me just validate the statement. Rag is basically a kind of database that is for AI workflows.
Who is the master of databases and data stores? whatever data engineers because you need to apply data modeling. You need to just store your data efficiently. You need to manage the data stores. That is why AI engineers who who are coming from the software engineering background are struggling a lot to manage rags because this is not their area. Rag is the area where data
professionals shine. But who is talking about rag in the world of data engineering? on lamba. So just trust this guy and just master this rag as soon as as possible because this will become very very very competitent competent in the near future. Why? Because see now you think
that if you are not learning rag and you are assuming that other person is not learning rag that is not true because other person is already learning rag. Okay. So you need to master rag and that is why I have created this free master class for you all so that you do not need to spend any money because I know what's going on in the market right now.
Just refer to this end toend masterass which is for rag for data engineers even for I would say for any data professional. Okay I would say for anyone but that is my perspective that if you are a data professional you can understand rag better than anyone else.
This is your flex area. You should flex in rag. Okay. So in this particular master class, let me talk about the prerequis. The good thing is there are no prerequis. You do not even need to have any ML knowledge, any LLM knowledge, any geni knowledge. Now some of you will say an Lambai we have watched your generative AI full course. We have watched your LLM full course. So um will it be beneficial
or if we have not watched those courses? Should we watch it? See in order to continue with this video even if you have not watched those courses it's fine you can continue with the with this video but just bookmark those courses because those are the backbone of AI. So you should complete and watch those courses as well. But just for this video, you do not need anything anything. Make sense? Okay. So without delaying, let me quickly discuss what we going to talk about in today's master class. Because I know your time is very very precious and you will be saying,
"Anlama, just tell us what we going to master today." And let me be very honest. This video is already recorded. So I'm now just recording this intro part because I know the quality of the video is super super high. Cool graphics, cool animations plus live demonstration using cool infographics and it is going to be a big one and the best thing I have already mentioned that all the nodes code example snippets what
did you say code examples that means we are going to code as well in this video yes you are going to code as well in this particular master class I will literally show you how to build a rag not just theoretically ally but practically as well it is going to be a master class like masterpiece. Okay. So let's quickly discuss what we're going to discuss in this particular video.
We are going to discuss all of these topics and I would say much more than these topics. So we will be starting with some basic understanding about embeddings and LLMs and then we will be just starting with our uh APIs and then we need to cover some topics such as called Yra because you need to understand why do we need to use rack.
We are talking about rack rack rack rag rack. You can see that we are covering all the fundamentals as well. It's not like jumping directly into the implementation. No, you will build strong fundamentals first. Then we then we're going to talk about vector database. Then we're going to talk about open AI. Then we're going to talk about loaders, chunking, splitters, rack storage. Basically, these are the techniques or basically steps we follow in order to build the rag application.
So you will be learning everything step by step. Then we're going to talk about different kinds of rag inmemory rack ped persisted rag reusable rag custom metadata rag multi-document rag. So everything is covered in this video. If you had a thought in the past that you will master rag one day this is the day.
This is the day I'm telling you bro. Just treat this video as a binge watch and you're going to enjoy it. Let me be very honest with you. you're going to enjoy it and the cool code snippets are like the gamechanging thing because you will be building rag on your own without any kind of knowledge. So that is amazing thing. Okay, then we going to talk about so much of useful things as
well in rag. Okay, are you excited? Are you excited? I don't know man because I'm putting so much of effort just for you so that you can learn AI data engineering almost for free. I'm trying my best to make it affordable to every single one. And in return, what I just ask a small comment, a like, and just hitting the subscribe button. That's it.
And that's all you need to do in return. That's it. Why are you not doing that? Just hit the subscribe button you all because I know my lovely da fam has already done that. But you now can become the part of my lovely da fam as well. Just hit the subscribe button.
just drop a lovely comment and hit uh like button as well obviously and you can just share this video with your friends as well because you should help others as well. I still remember like I just talked about just drop a lovely comment. So there was a guy who wrote lovely comment and he mentioned like because you said you drop a lovely comment so he just wrote lovely comment.
I love these people. Okay. So now let's actually get started with the video. But you know what? Before that I want to celebrate some of the happy positive moments before starting with this video.
So these are basically the hardworking people. Basically, these are my lovely da family members who have recently cracked the job offers and thank you so much for notifying me and so sorry if you just if I just missed your message or comment but do you know what I'm literally happy happy happy from the bottom of my heart and congratulations to all of you. You will
say just two bro hold on so we have more. So congratulations to all of you. Congratulations to all of you. Congratulations to all of you. Congratulations to all of you. Congratulations to all of you. And do we have more? Yeah, congratulations to all of you. So these people actually followed the content on this YouTube channel and dedicatedly I would say practiced along the videos and see this
is the result of their practice. This is the result of their hard work. They didn't question anything. They didn't um doubt. They simply followed and that's it. And that's you need to do in order to obviously crack any job offer. Make sense? So now I also want to see your comment here because I know you will also crack your dream of a dream job very very soon. I am with you. You will find all the videos, all the resources, everything on this channel. So just hit
the bell icon as well so that you will be notified on every video upload so that you do not miss any important concept because see I will be uploading so so so much of content so that you can become a competitive developer but it's your responsibility to consume that content as well right so I know you will do that I know I know I know so let's get started with the video and I oh there we have some more okay congratulations to all of these as well so now these are my handles if you want
to follow me if you want to just uh let's say connect with me. These are my handles and that's it. Nothing special. So let's get started with this amazing masterass and let's try to master rag in a step-by-step approach. Let's see. So now it's time to discuss rag masterass.
Okay. So rag is a very wide term to be honest. Okay, very wide term. It's equivalent to like I'm talking about term rag term is like in terms of um you can say use cases. So rag in today's era rag is equivalent to a database
literally yes rag is equivalent to a database in today's era. So if I ask you a question let's talk about database you'll be like wait what kind of database what is the use case that you want to perform with that database? um what is the end goal to create the database and so many other questions right similarly with rag as well rag earlier was just a term to talk about
databases and AI but not now rag is much more than that because we perform data modeling with rag we perform so many other things that we're going to discuss today okay so just for your kind information do not feel like rag is just a term that we need to learn learn in just 10 to 15 minutes and that's it.