NVIDIA RTX 5090 vs RTX 4090 for AI
Compare VRAM, hardware specifications, live Fluence pricing and measured LLM inference performance across matched configurations.
NVIDIA RTX 5090 vs RTX 4090 for AI
Compare VRAM, hardware specifications, live Fluence pricing and measured LLM inference performance across matched configurations.
AVAILABLE NOW
RTX 5090
32GB of GDDR7 VRAM for larger models, longer contexts and bigger batches.
VRAM
32GB
Architecture
Blackwell
Bandwidth
1,792 GB/s
Starting from
$1.20
/per hour
AVAILABLE NOW
RTX 4090
24GB of GDDR6X VRAM for lower-cost serving when the workload fits within 24GB.
VRAM
24GB
Architecture
Ada Lovelace
Bandwidth
1,008 GB/s
Starting from
$0.84
/per hour
AVAILABLE NOW
RTX 5090
32GB of GDDR7 VRAM for larger models, longer contexts and bigger batches.
VRAM
32GB
Architecture
Blackwell
Bandwidth
1,792 GB/s
Starting from
$1.20
/per hour
AVAILABLE NOW
RTX 4090
24GB of GDDR6X VRAM for lower-cost serving when the workload fits within 24GB.
VRAM
24GB
Architecture
Ada Lovelace
Bandwidth
1,008 GB/s
Starting from
$0.84
/per hour
RTX 5090 or RTX 4090?
The RTX 5090 adds 8GB of VRAM and 78% more memory bandwidth. That supports larger models, longer contexts and larger batches. The RTX 4090 can cost less when the workload stays within 24GB.
Specification
RTX 5090
RTX 4090
Architecture
Blackwell
Ada Lovelace
VRAM
32GB
24GB
Memory
GDDR7
GDDR6X
Memory bandwidth
1,792 GB/s
1,008 GB/s
Tensor Cores
5th generation
4th generation
Host interface
PCIe 5.0
PCIe 4.0
NVLink
No
No
RTX 5090 or RTX 4090?
The RTX 5090 adds 8GB of VRAM and 78% more memory bandwidth. That supports larger models, longer contexts and larger batches. The RTX 4090 can cost less when the workload stays within 24GB.
RTX 5090
RTX 4090
Architecture
Blackwell
Ada Lovelace
VRAM
32GB
24GB
Memory
GDDR7
GDDR6X
Memory bandwidth
1,792 GB/s
1,008 GB/s
Tensor Cores
5th generation
4th generation
Host interface
PCIe 5.0
PCIe 4.0
NVLink
No
No
Choose the right GPU for your workload
Match the GPU to the memory, throughput and runtime requirements of the workload you plan to run.
Workload
RTX 5090
RTX 4090
LLM inference
Longer contexts, larger batches and higher concurrency.
Lower-cost serving when 24GB and its measured capacity are sufficient.
Training and fine-tuning
More room for model weights, activations and larger batches.
Smaller models and memory-efficient LoRA or QLoRA runs.
Image and video generation
Larger images, longer clips and bigger batches.
Ada-based generation pipelines that remain within 24GB.
CUDA development and batch jobs
More capacity for demanding or growing workloads.
Development, testing and lower-cost batch processing.
Choose the right GPU for your workload
Match the GPU to the memory, throughput and runtime requirements of the workload you plan to run.
RTX 5090
RTX 4090
LLM inference
Longer contexts, larger batches and higher concurrency.
Lower-cost serving when 24GB and its measured capacity are sufficient.
Training and fine-tuning
More room for model weights, activations and larger batches.
Smaller models and memory-efficient LoRA or QLoRA runs.
Image and video generation
Larger images, longer clips and bigger batches.
Ada-based generation pipelines that remain within 24GB.
CUDA development and batch jobs
More capacity for demanding or growing workloads.
Development, testing and lower-cost batch processing.
Check your VRAM requirements
Estimate memory use for your model and serving configuration, then compare the result with 24GB and 32GB.
Will your model fit?
Estimate model weights, KV cache and runtime overhead before choosing a GPU.
7.9 GB
of 12 GB
Weights
5.0 GB
KV cache
2.1 GB
Overhead
0.8 GB
Free
4.1 GB
Check your VRAM requirements
Estimate memory use for your model and serving configuration, then compare the result with 24GB and 32GB.
Will your model fit?
Estimate model weights, KV cache and runtime overhead before choosing a GPU.
7.9 GB
of 12 GB
Weights
5.0 GB
KV cache
2.1 GB
Overhead
0.8 GB
Free
4.1 GB
RTX 5090 vs RTX 4090 benchmarks for LLM inference
Fluence tested both GPUs with the same Llama 3.1 8B Instruct workload to compare throughput, latency and cost as concurrency increased.
Benchmark result
RTX 5090
RTX 4090
Concurrent requests
24
16
Output throughput
1,435 tok/s
729.91 tok/s
Compliant throughput
11.20 req/s
5.70 req/s
P95 TTFT
220 ms
224.22 ms
P95 end-to-end latency
2,900 ms
2,769.76 ms
Cost per 1M output tokens
$0.232
$0.3197
Cost per 1K compliant requests
$0.030
$0.041
Maximum tested load is the highest concurrency that stayed within P95 TTFT of 500 ms and P95 end-to-end latency of 3,040 ms.
Results are medians across three instances and three repeats. Costs use captured rates of $1.20/hr for RTX 5090 and $0.84/hr for RTX 4090 and cover GPU compute only.
RTX 5090 vs RTX 4090 benchmarks for LLM inference
Fluence tested both GPUs with the same Llama 3.1 8B Instruct workload to compare throughput, latency and cost as concurrency increased.
RTX 5090
RTX 4090
Concurrent requests
24
16
Output throughput
1,435 tok/s
729.91 tok/s
Compliant throughput
11.20 req/s
5.70 req/s
P95 TTFT
220 ms
224.22 ms
P95 end-to-end latency
2,900 ms
2,769.76 ms
Cost per 1M output tokens
$0.232
$0.3197
Cost per 1K compliant requests
$0.030
$0.041
Maximum tested load is the highest concurrency that stayed within P95 TTFT of 500 ms and P95 end-to-end latency of 3,040 ms.
Results are medians across three instances and three repeats. Costs use captured rates of $1.20/hr for RTX 5090 and $0.84/hr for RTX 4090 and cover GPU compute only.
Choose how to access GPU capacity
Need available GPUs now or committed multi-GPU capacity?
Choose how to access GPU capacity
Need available GPUs now or committed multi-GPU capacity?
FAQ
Does a model smaller than 24GB automatically run on RTX 4090?
Can multiple RTX GPUs combine their VRAM?
Why might my results differ from Fluence benchmarks?
How are prices on this page updated?
Can I reproduce the benchmark?
FAQ
Does a model smaller than 24GB automatically run on RTX 4090?
Can multiple RTX GPUs combine their VRAM?
Why might my results differ from Fluence benchmarks?
How are prices on this page updated?
Can I reproduce the benchmark?
Be the first to know
Be the first to know