What are GPUs and why they are critical for AI

The graphics processing unit, a chip engineered to render pixels and push polygons across a screen, has quietly become the most critical piece of AI infrastructure.

Once confined to gaming rigs and workstations, the GPU now sits at the heart of AI data centres consuming entire city blocks' worth of electricity, training the large language models and generative AI systems reshaping industries from healthcare to finance.

Demand for these chips has grown so ferocious that Nvidia, their dominant manufacturer, has become the world's most valuable company; a remarkable ascent that signals just how central GPU compute has become to the AI arms race.

To understand why GPUs matter, it helps to contrast them with the processor most people are familiar with: the central processing unit, or CPU. A CPU is the general-purpose brain of a computer. It is fast, flexible, and built to handle a wide variety of tasks sequentially. It typically features between 4 and 64 high-performance cores, each designed to execute complex instructions one after another with remarkable speed.

A GPU, by contrast, is a specialist chip first commercialised in the late 1990s. It was built around a fundamentally different architecture, one featuring thousands of smaller, simpler cores designed to execute many calculations simultaneously. Where a CPU excels at deep, sequential reasoning, a GPU excels at breadth: running thousands of parallel operations at the same moment.

Nvidia's GeForce 256, launched in 1999, is widely credited as the first consumer GPU, though the term itself was coined by Nvidia as a marketing distinction. The chip offloaded the intensive task of rendering 3D graphics from the CPU, allowing games to look richer and run more smoothly than ever before.

For nearly a decade after their introduction, GPUs remained squarely in the domain of gaming and visual computing. Their ability to process pixel colours, lighting calculations, and geometric transformations in parallel made them extraordinarily well-suited to producing the real-time imagery that gamers demanded.

However, in the mid-2000s, researchers began realising that the GPU's parallel architecture could be applied far beyond graphics. In 2006, Nvidia released CUDA (Compute Unified Device Architecture), a programming platform that allowed developers to harness GPU cores for general-purpose computation, not just rendering.

The release of CUDA was transformative and scientists began using GPUs to model molecular interactions, simulate weather patterns, and process satellite imagery. The GPU had become a general-purpose parallel computer clothed in a graphics card.

The connection between GPU architecture and artificial intelligence is not coincidental but architectural. Training an AI model, particularly a neural network, involves performing an enormous number of mathematical operations simultaneously. At its core, deep learning is built upon matrix multiplication: multiplying and transforming vast grids of numbers across many layers to learn patterns in data.

This is precisely what GPUs were designed to do. A modern AI training job might require trillions of floating-point operations per second (FLOPS). Attempting to run such workloads on even the most powerful CPUs would be prohibitively slow. GPUs, with their thousands of parallelised cores, can complete these computations orders of magnitude faster.

The significance of this became dramatically clear in 2012, when researchers Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton used two Nvidia GPUs to train AlexNet, a deep neural network that decisively won the ImageNet visual recognition competition. The result sent shockwaves through the machine learning community. What had previously taken weeks on CPUs was accomplished in days on consumer GPUs. The AI era had found its engine.

Today, Nvidia dominates the AI chip market with its H100 and the more recent B200 Blackwell GPUs, purpose-built for the demands of large-scale AI training and inference. A single Nvidia H100 GPU, priced at roughly $30,000 (£24,000), can deliver 4 petaFLOPS of AI performance. Data centres now stack thousands of these chips together into clusters consuming tens of megawatts of power.

Competitors are accelerating development to challenge this dominance. AMD's Instinct MI300X has made inroads with cloud providers, while Google's Tensor Processing Units (TPUs) - custom silicon designed specifically for AI workloads - offer a proprietary alternative. Intel, too, is pushing into the market with its Gaudi accelerator line.

Yet for all the new entrants, GPUs remain the lingua franca of AI infrastructure. Their combination of parallel processing power, software maturity through platforms like CUDA, and ecosystem depth make them the default choice for training frontier models.

From rendering fantasy landscapes in video games to training systems that can write code, diagnose diseases, and generate art, GPU has been on a journey to become one of the most consequential pivots in the history of computing. What began as a specialist chip for consumer entertainment became the critical infrastructure of a technological revolution.

As AI models grow ever larger and more capable, the demand for GPU compute shows no sign of slowing. If AI is the mind of this new era, the GPU is very much its muscle.