Hey there 👋
You probably know about graphics cards or GPUs from that nerdy friend who owns a gaming PC.
But most people do not realize that every time you use ChatGPT, Claude, or any AI chatbot, you are using a GPU.
Actually, you are using thousands of them!
Those AI models are not running on regular computer processors. They are running on massive clusters of graphics cards in data centers.
The GPU in a gaming PC and the GPU powering ChatGPT are fundamentally the same technology. Just scaled up to big levels.
Everyone and their dog is talking about AI these days.
So, I thought it would be a nice idea to explore how GPUs came to the scene and why AI needs them.
What Even Is a GPU?
GPU stands for Graphics Processing Unit. Many call them graphics cards or video cards.
A GPU is a specialized computer chip designed to handle graphics. It looks pretty similar to a CPU but it’s quite different from it.
A GPU processes video and image data on your screen. Its primary purpose is to turn data into pixels on your screen that you can see.
An important thing about GPUs is that they are really good at doing lots of simple calculations at the same time.
If you’ve read our explainer article on CPUs, you know that CPUs have cores. A modern one typically has around 8 to 24 cores.
However, a modern GPU has thousands of cores. These cores are not very smart on their own. They are not even as broad in what they can do as a CPU core.
But because GPUs have so many of these cores, when combined they can do massive amounts of specific work in parallel.
Compare that to a CPU or Central Processing Unit. Your CPU has maybe 8 to 24 cores.
But each core is very smart and can do basically any calculation you do on it like watching videos, running spreadsheets, all that stuff.
A CPU is like having a few really smart people working on a problem. A GPU is like having thousands of average people working on the same problem at once.
For certain tasks the GPU approach is way faster and rendering a video game is one of those tasks. You need to calculate the color and position of millions of pixels 60 times per second.
That is a lot of simple repetitive math and it’s perfect for a GPU.
Your CPU could do it but it would be slow. Your GPU can do it instantly because it has thousands of cores all working in parallel.
This is why gaming PCs have dedicated graphics cards. The GPU handles all the graphics work while the CPU handles everything else.
How Are CPUs and GPUs Different?
Let me explain this with an example.
Imagine you need to paint a huge wall. You have two options.
Option one: hire one really skilled painter. This person is fast and precise. They can handle complex patterns and details. But there is only one of them.
Option two: hire 1000 average painters. They are not as skilled individually. But there are so many of them that they can cover the wall much faster.
The skilled painter is your CPU. The 1000 average painters are your GPU.
For complex tasks that require lots of decision making the CPU is better. Things like running different programs, browsing the web, doing spreadsheets and stuff.
For simple repetitive tasks that can be done in parallel the GPU is better. Graphics, video encoding, and, as it turns out, AI.
CPUs are optimized for serial processing. One task after another very quickly. GPUs are optimized for parallel processing doing thousands of tasks at the same time.
Intel’s Core i9 has 24 cores. Nvidia’s RTX 4090 has 16,384 CUDA cores. That is why GPUs are so much faster for specific workloads.
A Little History of Graphics Cards
GPUs became mainstream because of video games.
In the 1990s video games which were initially pretty simple started getting more complex. 3D graphics were becoming standard.
But CPUs were not great at rendering 3D graphics. They could do it but it was pretty slow.
So companies started making dedicated graphics cards with their own processors. These cards could handle all the graphics work while the CPU focused on everything else.
3dfx released the Voodoo Graphics card in 1996. It was the first consumer 3D graphics accelerator that really worked well.
Suddenly games like Quake and Unreal Tournament looked amazing. 3D graphics went from choppy and ugly to smooth and detailed.
Nvidia was founded in 1993. They released the RIVA 128 in 1997 and the TNT in 1998. These cards competed with 3dfx.
Then in 1999 Nvidia released the GeForce 256. This was the first GPU. Nvidia literally invented the term GPU with this card.
The GeForce 256 could do hardware transform and lighting. This offloaded even more work from the CPU to the graphics card.
Gaming GPUs got faster every year. More cores, higher clock speeds, better memory.
ATI competed with Nvidia through the 2000s with their Radeon cards. AMD bought ATI in 2006.
By the late 2000s gaming GPUs were extremely powerful parallel processors. Thousands of cores capable of trillions of calculations per second.
But researchers realized something.
Those same parallel processing capabilities that we use for games are also perfect for other forms of computing.
Then AI Happened
We can call 2012 the first major breakthrough in the current AI work that’s being done in front of us.
A team at the University of Toronto led by Geoffrey Hinton used GPUs to train a neural network called AlexNet in 2012.
AlexNet won the ImageNet competition that year by a huge margin. It was way more accurate than previous image recognition systems.
The key was using GPUs. Hinton’s team trained AlexNet on two Nvidia GTX 580 gaming GPUs.
Training neural networks (think of these as the backbone for ChatGPT) requires doing the same math operations just millions of times. Matrix multiplications mostly.
CPUs can do matrix multiplication but it is slow. GPUs can do it way faster because they have thousands of cores working in parallel.
AlexNet proved that GPUs could accelerate AI training by 10x to 100x compared to CPUs.
This kicked off the modern AI boom.
Researchers started using gaming GPUs for AI research. Nvidia GTX Titan, GTX 1080 Ti, RTX 2080 Ti became standard in AI labs.
Then in 2016 Google announced they built custom AI chips called TPUs. These were designed specifically for neural networks.
Nvidia responded by focusing more on AI. They released data center GPUs specifically for AI workloads.
The V100 in 2017. The A100 in 2020. The H100 in 2022.
These were not gaming cards. They were designed for training and running AI models in data centers.
The H100 costs around $30,000. Compare that to a gaming GPU which costs $500 to $2,000.
But data centers needed them and AI companies like OpenAI bought thousands of H100s.
When ChatGPT launched in November 2022, it ran on thousands of Nvidia A100 GPUs. Later OpenAI upgraded to H100s.
Nvidia’s current Blackwell chips have shipped in the millions, many as rack scale systems of 72 GPUs each.
Nvidia’s data center revenue was $4.3 billion in 2020. By 2025 it hit over $100 billion.
Almost all of that growth came from AI companies buying GPUs.
Why Claude Needs Thousands of GPUs
You might be wondering why an AI chatbot needs thousands of expensive GPUs.
The answer is that AI models like GPT-4 or Claude are pretty large. You get why we call them Large Language Models?
GPT-4 reportedly has around 1.7 trillion parameters. Parameters are the numbers that the model learns during training.
All those parameters have to fit in memory. A single GPU does not have enough memory for a model that large.
So AI companies split the model across many GPUs. Each GPU holds a piece of the model.
When you ask ChatGPT a question the work gets distributed across dozens or hundreds of GPUs working together.
This is called model parallelism. The model is too big for one chip so you split it across many chips.
There is also another reason. Inference.
Inference is the process of using a trained model to generate responses. When you send a message to ChatGPT that is inference.
ChatGPT has millions of users. At any given moment thousands of people are asking it questions.
Each question requires running the model. Each run takes a certain amount of GPU compute.
If you want to serve millions of users you need thousands of GPUs running inference simultaneously.
OpenAI reportedly has over 25,000 Nvidia GPUs running ChatGPT. Some estimates put it even higher.
Meta has over 100,000 H100 GPUs for training Llama models.
Google, Microsoft, Amazon all have massive GPU clusters for their AI services.
In 2023 and 2024 there was a global shortage of Nvidia H100 GPUs. Demand was so high that companies waited months for deliveries.
Jensen Huang (Nvidia’s CEO) joked that H100s were the new currency. Some companies bought H100s just to have them as assets.
This is why Nvidia’s market cap is sitting at more than $5 trillion as of writing this article. They have over 90% market share in AI GPUs!
AMD and Intel are trying to compete but Nvidia is untouchable in their software ecosystem which has been a goldmine for them.
Everyone’s talking about Nvidia these days but do you know that Nvidia doesn’t make their GPUs?
In fact their GPUs are manufactured by TSMC, a company so important that we may have a world war over it :/
CUDA and Nvidia’s Trillions
Nvidia did not set out to dominate AI. It started out as a GPU company just trying to survive for the next 30 days.
They became a gaming GPU company by years of hard work. AI, however was more of a side experiment for them to begin.
The irony is that Nvidia’s current AI revenue is so large that earnings from consumer GPUs seems like a rounding error in their financial reports.
Nvidia’s gaming revenue in 2025 was over $11 billion. Their data center AI revenue was over 100 billion.
AI is roughly 10x bigger than gaming for Nvidia.
But this happened because Nvidia made smart decisions pretty early on.
In 2006, Nvidia released CUDA. This was a programming platform that let developers use GPUs for general computing not just graphics.
Nvidia dropped more than a billion dollars into CUDA back in 2006. I covered that investment in quite detail in my Nvidia deep-dive article linked here.
At that time, most people thought CUDA was a waste of money. Why would anyone use a graphics card for non-graphics work?
But Nvidia kept investing in CUDA. They made it easy for researchers to write GPU-accelerated code.
When researchers started building neural networks they reached for Nvidia GPUs because CUDA made it easy.
By the time AI took off in the 2010s Nvidia had a 10 year head start on GPU computing software.
AMD had GPUs too. But their software stack was way behind Nvidia’s.
Intel tried to enter the GPU market with Xeon Phi and later discrete GPUs. But they could not compete with Nvidia’s ecosystem.
Every AI framework was built on CUDA. PyTorch, TensorFlow, JAX all assumed Nvidia GPUs.
Universities taught CUDA. Researchers published CUDA code. The entire AI ecosystem locked into Nvidia.
This is why Nvidia dominates AI hardware. Not because their chips are vastly better but because their software ecosystem is unmatched.
AMD’s MI300 GPU is competitive with Nvidia’s H100 on raw specs. But porting code from CUDA to AMD’s ROCm is painful.
That is why most people just stick to writing their code in CUDA and paying Nvidia’s premium.
GPUs Are Everywhere
You may be surprised but every time you use an AI chatbot you are using GPUs.
When you ask ChatGPT to write an email you are using Nvidia GPUs in OpenAI’s data center.
When you are planning your next startup with Claude (you know who you are :p), you are using Anthropic’s GPU clusters with thousands of Nvidia GPUs.
These GPUs cost millions of dollars to buy and millions more to operate. This is why some AI services charge good money.
There is a reason why basic subscriptions to Claude and ChatGPT cost more than $20. Running GPUs isn’t a poor man’s play.
Google’s Gemini models are different though because they are optimized to run efficiently on their TPUs.
What in the devil is a TPU you may ask?
Well, It’s too long of a topic to cover in this article. Let’s stick to GPUs for this one and if you guys want it, I can write a separate article on TPUs.
Most people think GPUs are just for gaming. And it’s not wrong. But in 2026 GPUs are everywhere.
Your phone has one. Your laptop has one. Even your car probably has one. And more importantly every AI service you use runs on GPUs in a data center somewhere.
Even when Netflix recommends shows or Spotify suggests music, those suggestions come from AI models running on GPUs.
Your bank uses GPUs to detect fraud. Your email uses GPUs to filter spam. Your phone uses GPUs to process photos.
GPUs have been powering most forms of AI for quite some time. We just started paying attention now.
Your Phone Has One Too
Every modern smartphone has a GPU built into the processor.
Apple’s iPhones have them. Snapdragon powered Android phones have them too.
These are not as powerful as desktop or data center GPUs. But they are still GPUs doing the same parallel processing.
When you play a mobile game the GPU renders the graphics.
When you take a photo the GPU processes the image. HDR, night mode, portrait mode all use GPU compute.
When you use Face ID or face unlock the GPU helps process the camera feed.
Some AI features on phones run locally on the GPU.
Apple’s on-device Siri processing uses the Neural Processing Unit (NPU) which is basically a specialized AI GPU.
If you’re scratching your head over what’s an NPU, my friend Mohib Ur Rehman dove deep into Neural Processing Units in his article linked here.
Google’s Pixel phones use Tensor chips with dedicated AI cores for features like Magic Eraser and Live Translate.
Samsung’s Galaxy phones use Exynos or Snapdragon chips both of which have GPU and NPU cores for AI.
The difference between phone GPUs and data center GPUs is just scale. Your phone GPU might have 6 cores. A data center GPU has 16,000 cores!
Who’s Coming for GPUs?
It doesn’t matter how much Jensen wants to sell you on his GPUs.
The truth is that GPUs are not the final answer for AI. They are just the best option we have right now.
And many companies are trying their best to change that.
Google is already using their custom chips called Tensor Processing Units (TPUs). They use it for training Gemini and doing most of their AI stuff.
Other companies are building custom AI chips too. Amazon has Trainium and Inferentia chips. Microsoft is working on their own AI accelerators.
And dozens of startups like Cerebras, Graphcore, and many others are building even more specialized AI chips.
All of them want a share from Nvidia’s complete dominance in AI.
It’s true that none of them have matched Nvidia’s combination of performance and compatibility ecosystem yet.
But that doesn’t mean it’s not possible. The next generation of GPUs is already in development.
Nvidia’s B100 and B200 GPUs were launched in 2024. AMD launched its MI350 in 2025 and just followed it with the MI400 series in mid-2026, its strongest challenge to Blackwell yet.
Intel’s also working on their Gaudi 3 AI accelerator.
So, for now Nvidia is winning. CUDA is too strong.
Every AI framework is basically written for CUDA. And almost everyone working on AI knows CUDA. That’s simply why people keep using it.
But that could change if companies like Google or Amazon can prove that custom chips are way better.
For us as end users, we may not be able to get our hands on these enterprise GPUs anytime soon (maybe never).
But it is cool to know that whenever you ask ChatGPT a question or generate an image, a GPU somewhere is doing the work.
In about 15 years, GPUs went from a specialized gaming chip to the foundation of AI.
Who knows what Nvidia will be like 10 years from now? CUDA may or may not be relevant.
But companies trying to compete is a good sign because we just don’t need more monopolies.
I hope this helped you understand what GPUs actually do and why they matter so much in the AI world.
If you enjoyed the read, a share to someone as curious as you would make my day.
