Data Centers, Debt & the Risks of AI’s Finances w/ Ed Zitron | The Weekly Show with Jon Stewart – Transcript

00:00On top of all of these other problems, we have the situation where hyperscalers are buying way more GPUs than they need, driving up the price of memory and thus the price of consumer electronics for everyone. And on top of that, they're lying, kind of, about how much capacity they have. They are trying to make everyone believe they're bringing gigawatts online when a data center can do many things. The data centers that people hate are not the ones that do

00:24streaming. These AI data centers are completely different beasts to anything that runs anything else. These things could all disappear tomorrow and the internet would function the mostly the same way because they only exist to do AI.

00:37>> Well, the AI thing was always it it it wasn't a necessary development in any way. You know, I I always felt like AI is just like Google that Google that's on Coke. Like, when you Google something, you say like, "Hey, man, is there a pizza place near me?"

00:53and it tells you. But when you say to AI, they're like, "Oh my god, that's such a great question. I love pizza. Pizza is so good. Did you know it started in Greece?" Like, it just tells you and you're just like, "Shut up." >> And sometimes it goes, "Yeah, DeMarco's is the best." Yeah, that's right. "Hey, man, I I need all your water. I need all your water."

01:09>> DeMarco's does not doesn't exist. >> doesn't exist and it makes up. >> >> Ladies and gentlemen, welcome to the Weekly Show podcast. My name is Jon Stewart. I am the host of the Weekly Show podcast. It is Tuesday, October 27th and excuse me, I'm going to stand back from the mic for just one moment.

01:30Can you hear the people sing? Singing a song. It's It's the election. It's This is the last time that that we will speak before the election and I just want to urge everybody out there to please, please, for the love of God, get out there and vote and express your political opinion. And I'd like to frame it as a however you'll vote, but I don't

01:54mean that. I mean it as please send a message to this reckless show and of administration uh that enough is enough uh that we have had it that it is time for someone to step up and provide uh I believe the framers referred to them as checks and balances on this uh utterly imperious administration. So, that is that is the spiel for today. We are not talking about the election

02:21today. We will be talking about other [snorts] doomsday scenarios uh financial ones in particular about AI. I know we've talked a a little bit on the program about catastrophic outcomes of AI in terms of bad actors or in terms of the AI itself turning on us.

02:40This deals more with uh the real world implications of AI economically and financially and we're delighted uh to have a guest today to talk about that. So, let's just get to it. >> >> Ladies and gentlemen, we're delighted to be joined by Ed Zitron. He's the CEO of Easy Primary Research. He's the writer of Where's Your Ed At newsletter but more importantly has a great understanding of I think uh the

03:08economics of AI and Ed, welcome. >> Thanks for having me. >> Uh it's it's an absolute pleasure. I want to ask you this and I apologize in advance. This is going to be slightly remedial in some regard. So, I wanted you to talk about you know, we've heard a lot about AI in terms of doomsday scenarios uh bad actor gets a hold of it, builds

03:32their own biohazard or uh the agents break out and hack into all of the sensitive data and they or water systems things like that or the reverse it all works and we're reduced to uh NPCs in a world that we have no agency in. I want to talk to you about the sort of the more economic imbalances

03:58of the AI world. Does that make sense? >> Absolutely. >> All right. Uh can you give us to start with kind of a little bit of the playing field of what is the AI economic ecosystem?

04:15Chips, software, data set. What What are we dealing with when we talk about the AI economy? >> So, at the top of the stock market, largest company in the stock market is Nvidia. They make something called a GPU. Technically means graphics processing unit. This isn't for graphics. It's because they're really good at parallel processing. So, basically, doing one thing a bunch of times. Nevertheless, you need those chips to do two things. One, to train an AI model and two, to run inference, which is just

04:43creating the outputs. >> When you say creating outputs, you mean so when you put in a question, the answer that comes to you is the output. >> Yes, exactly that. >> I see. Okay. >> And then it gets more complex with bigger models and bigger tasks, but nevertheless, that's the basics.

04:58>> these chips are the GPUs and they run the entire ecosystem. >> Yes. So, and Nvidia up until the last few years have basically had an effective monopoly on doing this because they have this thing called CUDA, which is just underlying software libraries and ways of running software on graphics processors. That This meant that they were they are effectively the company behind the AI boom. Now, they sell these GPUs, which are very expensive. They're

05:2550,000, 60,000 a piece. Usually buy them in racks that are millions of dollars each. And you need thousands of them, hundreds of thousands in some cases to do these big data centers. But Nvidia, pretty much the only vendor selling them. There's technically AMD.

05:39Technically, Broadcom has AI chips, too, but Nvidia's the top dog. Their customers are pretty much the largest companies in the world. Microsoft, Google, Meta, Amazon, Oracle, SpaceX, now, I guess. And they have a very small customer base, but that customer base has so much money that they've been able to sink 30 to 40 to 50 billion dollars a quarter into these AI chips and into AI data centers. Now,

06:04the problem is is that these things keep getting more expensive and also take forever to install. But, for the most part, Nvidia has got large because everybody has said AI is going to be the next big thing. We need to buy as many of these as possible.

06:20Not really Not really much proof yet that that's actually the case or at least anything close to a justification for the trillion plus dollars that have been invested in these GPUs and the data centers for them to go into. I can kind of stop there. There are various levels to fan out from there.

06:38>> Right. But, so the the general idea is you have this one company, Nvidia. Now, does Nvidia are they supplying it to everyone? Are we including China in that equation or we saying that's merely the the United States and maybe other sort of uh AI startup areas?

06:59>> So, technically China has a company called Huawei which makes their own chips, but Nvidia GPUs have been making it over there for years. There was a big story at the end of last year from Bloomberg that there's this company called Megaspeed, which is called a Neo Cloud. It just means a company that buys chips and installs them and rents them out, where they appeared to be, allegedly, a front for a Chinese data center operation. Supermicro, an AI server company,

07:25was also had one of their co-founders literally that using a hair dryer to get the serial numbers off. The It's so what? >> It's like It's like weapons. >> Here's the thing. In this era is so grim and soulless and joyless, it's nice to have some goofy high jinks for >> No, I think it is.

07:45>> So, Nvidia chips are making over there, but they have far less and China has had to be a little more deft with how they're building these things. But, for the most part, Nvidia is selling GPUs to everyone everywhere throughout the world. So, Nvidia basically controls everything. This means that Nvidia is incredibly rich. They're printing money.

08:04It means that Nvidia has now become the largest supplier to Microsoft, Google, Amazon, Meta, SpaceX. The problem is because these data centers are not actually generating enough revenue or really any revenue. They They don't dis- None of these companies, other than SpaceX, disclose their actual AI revenues.

08:23Everyone's had to start taking on debt. They've had to lay people off. Meta's doing a $95 billion share sale. Oracle had to do a $50 billion one, facing a downgrade potentially, all because despite all of the big promises of AI, the economic payoff is not that Torsten Slok, head of chief economist at Apollo, said the other day that there has been no payoff from AI.

08:44>> Well, that that gets to the heart of kind of the maybe the absurd contradiction in all this is that so you have this ecosystem built around these Nvidia chips that cost millions of dollars and these and what it does is it creates, I guess, compute. Everybody's looking for compute, but compute is very expensive.

09:06And apparently, and and you tell me if this is incorrect, but the people that use compute, the customers, don't pay anywhere near how much the compute costs to build and run. In other words, uh you're you're buying something and you're getting it at ridiculous discounts and the company is taking on

09:33that excess cost. >> To an extent. So, this is actually The thing is, you're both wrong and we don't know, but also right at the same time. >> I will take that. I will take that as a win. >> Let me explain what I mean. We don't actually know how much it costs to run these things.

09:48>> Oh, we don't? >> We don't know how much it costs. We have a lot of estimates. A lot of AI people online have come up with their guesses, but we truly don't know. We have these company called Silicon Valley that does an indices of how much you can get per hour for a GPU. And these are always going up and people say, "Wow, it must be They must be making the money back."

10:07We actually truly don't know the operational cost, but also even those indexes of these prices don't really reflect what people are paying long-term because they're mostly based off of spot prices, which just means what it what it would cost to rent this right now versus if I buy it for a year.