Nvidia's main AI accelerators have progressed from the A100 through H100 and H200 to Blackwell, Blackwell Ultra and Vera Rubin. The sequence looks simple until product names from several levels are mixed together. H100 and H200 are GPU products. Hopper is their architecture. GB200 combines Grace CPUs and Blackwell GPUs in a larger platform. GB300 NVL72 is a rack-scale system, not a single graphics card.
The correct choice depends on the model, memory requirement, precision, system topology, software stack, service target and total cost. A newer accelerator can provide substantial advantages, but availability, power density and migration work may make an earlier generation more practical for a specific deployment.
This article owns the Nvidia hardware-generation comparison. For the category-level explanation of parallel processing, CPUs and GPUs, start with what GPUs are and why they are critical for AI. For the complete processors, networking, systems and software map, use our Nvidia AI guide. The Nvidia CUDA guide covers the programming environment that turns the hardware into a usable platform.
Nvidia AI GPU generations at a glance
| Generation | Main AI products | Memory headline | Best understood as | Current role in 2026 |
| Ampere | A100 | Commonly 40 GB or 80 GB HBM configurations | The generation that scaled much of the pre-generative and early generative-AI build-out | Still widely deployed and useful for compatible workloads, but no longer the newest platform |
| Hopper | H100 | Up to 80 GB on standard H100 configurations; H100 NVL pairs larger-memory GPUs | Major step for transformer training and inference | Mature, broadly available and still central in clouds and installed clusters |
| Hopper refresh | H200 | 141 GB HBM3e and 4.8 TB/s memory bandwidth on the standard data-centre product | Hopper compute with substantially more and faster memory | Strong option for memory-constrained inference and HPC without moving to Blackwell |
| Blackwell | B100, B200, GB200 | Larger HBM capacity and a rack-scale Grace Blackwell design | A platform built around low-precision AI and large scale-up domains | Shipping across cloud, OEM and rack-scale configurations |
| Blackwell Ultra | B300, GB300 | Higher memory and inference capacity than the first Blackwell wave | Blackwell optimized further for reasoning, agents and dense inference | Shipping in systems such as DGX B300 and GB300 NVL72 |
| Vera Rubin | Rubin GPU and Vera CPU platform | New generation of compute, networking and memory systems | Nvidia's post-Blackwell platform for large-scale reasoning and agentic AI | In production according to Nvidia, with partner systems rolling out in the second half of 2026 |
Memory, form factor and availability differ by card, module, server, rack and cloud provider. Do not use the table as a procurement specification. Verify the exact system SKU.
Architecture, GPU, superchip and rack are different things
A useful comparison starts by separating the naming levels.
Architecture
An architecture is a generation of design choices. Examples are Ampere, Hopper, Blackwell and Rubin. One architecture can appear in multiple products.
GPU product
A GPU product is an accelerator such as A100, H100, H200, B200 or B300. Products based on one architecture can have different memory, power and form factors.
Module or superchip
Nvidia can package CPUs and GPUs into closely connected modules. Grace Hopper and Grace Blackwell configurations combine general-purpose CPU processing with accelerated compute and high-bandwidth links.
Baseboard and server platform
HGX is a platform used by server manufacturers and clouds to build multi-GPU systems. DGX is Nvidia's own integrated system family. Two servers using the same GPU can still differ in CPU, network, storage, firmware, cooling and support.
Rack-scale system
Products such as GB200 NVL72 and GB300 NVL72 combine dozens of GPUs, Grace CPUs, NVLink switching and liquid cooling across a rack. They should be compared with another rack-scale system, not with the purchase price or power draw of one accelerator.
Cloud instance
A cloud provider exposes a defined slice or cluster configuration. The instance name may refer to a virtual machine, one host, a multi-node cluster or a managed service. Cloud performance also depends on network and storage choices outside the GPU.
Why GPU memory matters
AI discussions often focus on FLOPS, but memory can determine whether a workload runs at all.
A model's weights, activations, attention cache, optimizer state and temporary tensors consume accelerator memory. The requirement changes with:
- model size and number of active parameters;
- numerical precision;
- context length;
- batch size and concurrent users;
- training versus inference;
- optimizer and fine-tuning method;
- tensor, pipeline and expert parallelism;
- and the efficiency of the serving software.
More memory can reduce the need to split a model across devices. Faster memory bandwidth helps feed the compute units. Those advantages can improve latency and utilization, but they do not guarantee a proportionate speed increase. A workload bottlenecked by networking, storage, CPU preprocessing or inefficient kernels may gain less.
Nvidia A100: Ampere's lasting installed base
The A100 was Nvidia's flagship data-centre accelerator before Hopper. It became a standard platform for deep learning, scientific computing and early large-language-model development.
Common configurations use 40 GB or 80 GB of HBM memory, with PCIe and SXM form factors. Multi-instance GPU technology can partition an A100 into smaller isolated instances, which can improve utilization for compatible workloads.
Where A100 still makes sense
A100 remains relevant when:
- the model and software have already been validated on it;
- cloud or used capacity is available at an attractive price;
- the workload does not need the memory or transformer-specific improvements of later generations;
- a team values operational stability over the newest hardware;
- or a smaller training, fine-tuning or inference job can achieve its service target economically.
The mistake is to treat an older generation as automatically cheap. A slower accelerator can cost more per completed workload if it needs more devices or runs for much longer. Compare the complete job.
Nvidia H100: Hopper becomes the generative-AI standard
H100 is based on the Hopper architecture and became the defining accelerator of the first large generative-AI infrastructure wave.
Nvidia added fourth-generation Tensor Cores and a Transformer Engine designed to use lower precision while preserving model quality. The platform also strengthened confidential-computing and virtualization capabilities, while SXM systems used high-bandwidth NVLink connections.
Nvidia's current H100 product page lists an 80 GB H100 SXM configuration and an H100 NVL configuration with 94 GB per GPU. Exact specifications depend on form factor.
H100 strengths
- broad cloud and OEM availability;
- mature CUDA and framework support;
- extensive benchmark and operational experience;
- strong training and inference performance;
- and a large installed base that makes skills and tooling easier to find.
H100 limitations
- 80 GB memory can be restrictive for large inference workloads or long contexts;
- cloud prices and supply can remain high;
- dense clusters require expensive networking and facilities;
- and newer products can deliver better performance or cost per token when the software is ready.
H100 is not obsolete because Blackwell exists. It is a mature production platform whose value now depends heavily on price and workload fit.
Nvidia H200: more memory on Hopper
H200 uses the Hopper architecture but adds 141 GB of HBM3e memory with 4.8 TB/s of memory bandwidth, according to Nvidia's H200 specifications.
That change matters for workloads constrained by memory capacity or bandwidth. A larger model or attention cache may fit on fewer accelerators, reducing communication and operational complexity.
H200 can be attractive for
- large-model inference;
- retrieval and reasoning workloads with long contexts;
- HPC workloads that move large datasets through accelerator memory;
- organizations that want more memory while staying within a Hopper-compatible operational environment;
- and deployments where H200 NVL availability or bundled software support is favorable.
What H200 is not
H200 is not a new architecture after Hopper. It is a memory-focused product evolution within that generation. Blackwell adds broader architectural and system changes.
Nvidia Blackwell: B100, B200 and GB200
Blackwell is Nvidia's successor to Hopper. It was designed around generative AI, lower-precision computation, large models and data-centre-scale systems.
Important Blackwell concepts include:
- second-generation Transformer Engine capabilities;
- support for low-precision formats including FP4 in suitable workloads;
- greater memory capacity and bandwidth;
- NVLink scale-up among large numbers of accelerators;
- Grace CPU integration;
- and rack-scale liquid-cooled configurations.
B100 and B200
B100 and B200 are Blackwell GPU products for data-centre systems. Exact deployment differs across HGX, DGX, OEM and cloud platforms.
GB200
GB200 refers to a Grace Blackwell configuration integrating Grace CPUs with Blackwell GPUs. GB200 NVL72 is a complete rack-scale system with 72 Blackwell GPUs and 36 Grace CPUs connected through NVLink.
This distinction is essential. A customer evaluating a GB200 NVL72 is buying a tightly integrated rack with substantial power, cooling and networking requirements—not simply 72 standalone accelerator cards.
Blackwell Ultra: B300 and GB300
Blackwell Ultra extends Blackwell for reasoning, agentic and physical-AI workloads that can require larger memory, more repeated inference steps and higher token throughput.
DGX B300
Nvidia's DGX B300 integrates eight Blackwell Ultra GPUs in a 10U system. Nvidia lists approximately 2.1 TB of total GPU memory and a power profile around 14 kW for the system. Facility and rack design must account for the complete server, not only accelerator thermal design power.
GB300 NVL72
Nvidia's current GB300 NVL72 page describes a liquid-cooled rack with:
- 72 Blackwell Ultra GPUs;
- 36 Grace CPUs;
- 130 TB/s of NVLink bandwidth;
- approximately 20 TB of GPU memory;
- and 37 TB of fast memory across the rack.
Those are Nvidia product specifications. Performance charts on a vendor page should be treated as vendor results with workload and configuration assumptions, not as universal independent benchmarks.
Who should consider Blackwell Ultra?
The platform is aimed at organizations operating very large models, high-volume inference, long-context systems, reasoning services, agentic workloads and dense training infrastructure. It is not automatically the best economic option for a small model or lightly utilized private cluster.
Nvidia Vera Rubin: the generation after Blackwell
Vera Rubin is Nvidia's next platform generation after Blackwell. The platform combines Rubin GPUs, Vera CPUs, next-generation networking, BlueField infrastructure and rack-scale systems.
Nvidia announced in 2026 that the platform's principal chips were in production and that partner systems would arrive in the second half of the year. Availability will differ by product, cloud, partner and region.
Rubin illustrates Nvidia's shift from announcing one GPU to announcing a platform containing multiple chips and systems. The relevant comparison will be rack-level performance, energy, memory, networking, software and cost—not only one GPU's peak compute.
Do not buy a roadmap
A roadmap is useful for capacity planning, but it is not delivered infrastructure. Before committing to a future generation, verify:
- order and delivery dates;
- cloud-region availability;
- production software support;
- framework and container compatibility;
- facility readiness;
- service and replacement commitments;
- and whether the expected workload exists when the system arrives.
The importance of Tensor Cores and numerical precision
Tensor Cores accelerate matrix operations used by neural networks. Successive generations support additional numerical formats and improved algorithms.
Lower precision can increase throughput and reduce memory use. Common formats include FP32, TF32, FP16, BF16, FP8 and FP4 or related low-bit formats.
Lower is not automatically better. The correct format depends on:
- training stability;
- model architecture;
- acceptable accuracy loss;
- quantization method;
- hardware and software support;
- and whether the bottleneck is compute or memory.
A vendor may publish dramatic gains at FP4 compared with an earlier system using a different precision or serving stack. Buyers should inspect the complete test and verify that the model quality and workload match their own requirements.
NVLink, NVSwitch and why scaling matters
One accelerator can only hold and process so much. Large systems partition models and workloads across devices.
NVLink provides high-bandwidth point-to-point or switched communication among Nvidia processors. NVSwitch expands the scale-up domain so more accelerators can communicate with high aggregate bandwidth.
This reduces—but does not eliminate—the cost of distributed work. Software still has to partition the model, schedule communication and keep devices synchronized. At cluster scale, InfiniBand or Ethernet networking connects servers and racks.
The practical result is that a GPU decision becomes a system-topology decision. Our DGX and AI factory guide covers that larger architecture.
PCIe versus SXM and integrated systems
PCIe accelerators
PCIe cards can fit standard compatible servers and may provide more flexible deployment. Their power and interconnect characteristics can differ from high-end integrated modules.
SXM modules
SXM is used in tightly integrated multi-GPU platforms with high-bandwidth NVLink and greater power delivery. It can offer higher performance but requires a purpose-built server.
NVL and rack-scale products
NVL configurations combine multiple GPUs or complete racks designed around high-speed interconnects. The term can refer to very different scales, so verify the exact SKU.
Do not assume a benchmark for an SXM eight-GPU system describes a single PCIe card. Form factor, power limit and network topology can change the result materially.
Which Nvidia GPU is best for AI?
There is no universal winner. Use the workload and procurement context.
| Situation | Sensible starting point |
| Existing validated Ampere environment with modest requirements | Price A100 capacity against the cost of migrating and the runtime of newer hardware |
| Mature training or inference with broad availability needs | H100 remains a common baseline |
| Hopper-compatible deployment constrained by memory | Evaluate H200 |
| New large-scale training and inference platform | Compare Blackwell or Blackwell Ultra systems with current alternatives |
| Dense reasoning and agentic inference | Evaluate Blackwell Ultra and current serving software |
| Capacity arriving later in 2026 or beyond | Include Vera Rubin in roadmap planning, but contract against actual delivery and support |
| Small or intermittent workload | Start with cloud rental, hosted APIs or smaller accelerators rather than owning a rack |
| Need to reduce platform dependence | Benchmark AMD or custom cloud accelerators as well as Nvidia |
How to benchmark an AI GPU properly
- Use the exact model or a representative architecture.
- Set the same precision and quality target.
- Use realistic input and output lengths.
- Match batch size and concurrency to the service.
- Include framework, compiler, library and serving versions.
- Measure warm-up separately from steady state.
- Record latency percentiles, not only average throughput.
- Include network and storage when the workload is distributed.
- Measure utilization and energy.
- Calculate cost per accepted result, not only tokens per second.
- Test failure recovery, checkpointing and operational tooling.
- Repeat after software optimization; first-run results can misrepresent a platform.
Cloud versus owned Nvidia GPUs
Cloud advantages
- faster access;
- no facility construction;
- flexible scaling;
- integrated storage and networking;
- easier experimentation across generations.
Cloud disadvantages
- variable and potentially high long-run cost;
- quotas and regional scarcity;
- data-egress and surrounding service charges;
- less control over exact hardware and maintenance;
- provider-specific networking and orchestration.
Owned or dedicated infrastructure advantages
- control over utilization, scheduling and data location;
- predictable capacity;
- potentially lower unit cost at sustained high utilization;
- deeper system customization.
Owned infrastructure disadvantages
- capital commitment;
- depreciation risk during rapid product cycles;
- power and cooling work;
- operations and spare-parts requirements;
- and the possibility of low utilization.
Nvidia versus AMD at the GPU level
AMD's Instinct accelerators compete directly with Nvidia data-centre GPUs. Current AMD products offer large HBM capacity, high memory bandwidth and an increasingly complete rack-scale and software platform.
The decision cannot be made from GPU specifications alone. CUDA and ROCm compatibility, cloud availability, network topology, framework support, procurement, staff skills and total system cost can outweigh a single peak number. Use our dedicated Nvidia versus AMD for AI comparison for the full decision.
Common buying mistakes
Comparing unlike products
A GPU card, eight-accelerator server and rack-scale system are not equivalent.
Buying theoretical peak performance
Peak FLOPS at a low precision do not describe model quality, memory pressure, communication or serving latency.
Ignoring software versions
A new GPU can underperform when the framework, kernels or container are not optimized for it.
Underestimating power and cooling
High-density systems can require liquid cooling, new power distribution and facility changes.
Assuming supply equals availability
A product can be “shipping” while the desired configuration, region or cloud quota remains unavailable.
Failing to price the whole cluster
Networking, storage, CPUs, racks, support, energy and staff belong in the comparison.
Frequently asked questions
What is Nvidia's best GPU for AI?
For the largest current deployments, Blackwell and Blackwell Ultra are Nvidia's leading shipping platforms, while Vera Rubin is entering rollout. The best economic choice still depends on workload, software, availability and total cost.
Is H200 better than H100?
H200 provides much more and faster memory, which can materially improve memory-constrained workloads. It will not improve every task by the same amount, and price and availability matter.
Is Blackwell a GPU?
Blackwell is an architecture. B100 and B200 are GPU products, while GB200 refers to a Grace Blackwell configuration and GB200 NVL72 is a rack-scale system.
What is Blackwell Ultra?
Blackwell Ultra is an enhanced Blackwell platform for reasoning, agents and physical AI, appearing in products such as B300, DGX B300 and GB300 NVL72.
Is Vera Rubin available?
Nvidia says the platform's chips are in production and partner systems are rolling out in the second half of 2026. Verify the exact product, provider, region and delivery date.
How much memory does H200 have?
Nvidia lists 141 GB of HBM3e with 4.8 TB/s of memory bandwidth for the standard H200 data-centre accelerator.
Can one H100 run a large language model?
It depends on model size, precision, context and serving method. Many large models require quantization, offloading or multiple accelerators.
Can gaming RTX GPUs train AI models?
Yes, for many local experiments and smaller workloads. Consumer cards differ in memory, reliability, support, virtualization, interconnect and data-centre qualification.
What is the difference between DGX and HGX?
HGX is a platform and baseboard architecture used by Nvidia partners. DGX is Nvidia's own integrated system family built around its compute and software stack.
Do Nvidia GPUs work without CUDA?
They can run graphics and other supported workloads, but most Nvidia AI acceleration relies directly or indirectly on CUDA drivers, libraries and software.
Bottom line
Nvidia's AI GPU roadmap is now a system roadmap. A100 established a large accelerated-computing base. H100 made Hopper the standard of the first generative-AI expansion. H200 added more memory. Blackwell moved the design toward lower-precision and rack-scale computing. Blackwell Ultra targets heavier inference and reasoning. Vera Rubin extends the platform again across compute, networking and software.
The newest generation is not automatically the best purchase. Match memory, precision, latency, throughput, topology, power, software and availability to the real workload. Compare equivalent systems and calculate the complete cost per useful result.
byDavid Porter
Loading