Why GPU prices spike during AI compute shortages

GPU prices spike when AI demand outpaces the memory, chip packaging, and power new GPUs need. In 2026, NVIDIA’s RTX PRO 6000 list price rose from $8,565 to $16,000, and B200 rentals jumped 79% in three months. Yet H100 rentals fell 64% to 75% from their 2024 peak.

This article explains what a GPU price spike is, which prices moved in 2026, and why shortages push them up. It also covers who drives demand, when prices ease, how GPU prices in India compare, and how teams can plan around them.

What is a GPU price spike?

A GPU price spike is a sharp rise in what buyers pay to rent or own a GPU, usually within weeks or months. It shows up in two places: hourly cloud rates and hardware list prices. The two can move at different times and by different amounts.

Contract type matters too. Spot and on-demand rates track the market daily, while reserved contracts lock a price for a year or more. In 2026, reserved H100 prices rose about 40% while on-demand prices fell, because on-demand capacity was sold out.

GPU prices in 2026

Cloud GPU rental prices are up 13% over 12 months across 21 models, with 16 rising and 5 falling. The GPU prices chart below shows the biggest moves. Rental figures are in US dollars per GPU-hour.

Note: Trackers measure different things, so the same GPU can show different prices. On 7 October, one list-price median put the H100 at $4.20, while trade-based indexes showed about $2.81 to $2.86.

Why do GPU prices spike during AI compute shortages?

GPU prices spike because demand rises fast while supply cannot. New supply depends on memory, packaging, and power that take quarters or years to expand. With supply fixed, price absorbs the extra demand.

Three forces stack up. Demand jumps when new AI products launch. Supply is capped by a few scarce parts. Contracts then lock capacity away, so less reaches the open market. Prices rise first where capacity is tightest: newest GPUs, high-memory cards and reserved contracts.

The demand behind GPU prices

Three groups drive most of the demand: the biggest cloud providers, AI labs, and AI products that run all day. Each buys GPUs for a different reason, and together they leave little spare capacity for everyone else.

Cloud providers buy to build capacity: Capex means money spent on data centres, servers and chips. The four largest US cloud providers have guided about $730 billion of capex for 2026, up 78% from 2025. Much of it goes to GPUs, so their orders set the pace for the whole market.

Borrowed money keeps orders coming: Cash flow no longer covers the full bill. Debt funded 32% of incremental capex by mid-2026, up from 9% in FY24. In our reading, buyers who borrow to build are less likely to pause when GPU prices rise.

Agents turn a one-time cost into a daily one: Training builds a model once. Inference runs it every time someone asks a question, and an AI agent may run many steps for a single task. SemiAnalysis ties the 2026 crunch to reasoning and agentic models, and Gartner expects inference spending of $23.3 billion to pass training at $19 billion in 2026.

The result is steady, all-day demand on top of one-time build-outs. That is why rental prices can rise even while new GPUs ship.

Bottlenecks in the GPU Supply Chain 

Supply is stuck at three points: memory, chip packaging and power. Because a GPU needs all three, the slowest one sets the pace. Each takes years to expand. Let’s look at these three in detail below:

Memory is the first limit

Memory is the first limit because HBM, the fast memory stacked beside the GPU, is sold out. A single B200 needs eight HBM3E stacks, and only Samsung, SK Hynix and Micron make HBM in volume. Micron said its HBM capacity for 2025 and 2026 is fully booked.

Scarcity has raised prices. Samsung and SK Hynix lifted HBM3E prices by nearly 20% for 2026. Memory now makes up 30% to 40% of an AI accelerator’s cost, up from under 20% two generations ago.

The same shortage reaches graphics cards. NVIDIA raised prices a third time in 2026, by 20% to 30% on some cards. Cards with more VRAM, such as 16GB models, carry more memory chips, so they are more exposed than 4GB or 8GB cards. The RTX PRO 6000 uses 32 memory modules, so each memory price rise is multiplied by 32.

Packaging caps how many GPUs ship

Packaging caps output because every GPU must be joined to its memory by TSMC’s CoWoS process. TSMC is the main supplier, so queues form fast. Capacity grew from about 35,000 wafers a month in late 2024 toward 125,000 to 130,000 by the end of 2026.

Even so, lead times run 52 to 78 weeks, and NVIDIA has booked roughly 60% of 2026 output. In our reading, when one buyer holds most of the slots, others wait, and waiting buyers bid up prices in the open market.

Output is also slow to add. A gap of about 10% remains after a ninefold rise, and new packaging lines take 12 to 18 months to qualify.

Power slows new data centres

Power slows new data centres because a finished GPU still needs a grid connection to run. Chips can be built in months, but grid approvals take years. Gartner projects that 40% of AI data centres will be power-constrained by 2027.

Scale makes it harder. A site with 1,000 GPUs needs about 1.76 MW, which calls for dedicated substation capacity. Approval for that kind of connection takes 24 to 36 months in major US and European markets.

When do GPU prices spike, and when do they fall?

GPU prices spike when new demand lands before new supply, and they fall when supply catches up. The record from 2024 to 2026 shows four phases.

From the above table reading, each fall followed new supply, and each rise followed a new kind of demand or a new bottleneck. The shortage has moved from GPUs to memory and power, which is why spikes keep returning.

Where do GPU prices in India stand?

GPU prices in India follow global memory and supply costs, but rupee billing and local capacity change what buyers pay. NVIDIA GPU prices in India are tied to US dollar list prices, so Indian buyers also carry rupee moves, at about ₹95 to the dollar on 13 October.

Government compute is the cheapest route. IndiaAI had over 45,000 GPUs onboarded by June, with average portal rates near ₹67 per GPU-hour and up to 40% subsidy for eligible users. Commercial rates are higher. 

How CloudPe’s rates compare

As of 14 October 2026, CloudPe lists the L4 at ₹35.86 an hour, the RTX PRO 6000 at ₹185.56, and the H200 at ₹410. Billing is in INR from a Mumbai Tier-4 data centre, with no data-transfer surcharges within India. The table converts rates at ₹95 to the dollar.

CloudPe sits well below hyperscaler list prices and close to or below the market median. It is not the lowest everywhere. 

These comparisons need like-for-like terms. Google Cloud figures are US-region on-demand list prices, and its H200 rate comes from an 8-GPU node. CloudPe rates exclude GST. Live rates change with memory and supply costs, so the pricing page is the source of record.

Strategies to Manage GPU Cost Volatility 

Teams plan best by matching the GPU to the model, the billing plan to the load, and tracking cost per task. Four steps cover most cases.

1. Right-sizing: A model that fits in 24 GB can run on an L4. A 70B model at FP8 needs about 70 GB, which fits on a 96 GB RTX PRO 6000. Unused memory is the easiest cost to cut.
2. Billing fit: Steady workloads suit monthly or yearly plans. CloudPe’s L4 page lists a 12% saving on annual plans.
3. Cost per task, not price per hour: Gartner predicts that 60% of Global 500 companies will embed AI FinOps controls at inference by 2028. Finance teams then see the cost of one finished task.
4. Data location: Data that stays in India supports DPDP, RBI and SEBI data localisation needs. Each buyer should confirm which rules apply to its own data.

What should buyers watch next in GPU prices?

Buyers should watch whether supply or demand is moving first; look at the four signals mentioned below:

Packaging capacity shows how many GPUs can ship: Packaging joins the GPU chip to its memory, so it caps output. TSMC’s CoWoS capacity is set to reach 120,000 to 140,000 wafers a month by the end of 2026, with partner packagers adding 50,000 to 60,000. More output reaching buyers beyond the largest customers would ease pressure.

Memory supply shows how fast HBM catches up: GPUs cannot ship without HBM, and it is sold out. Relief is not expected before late 2027 at the earliest. Falling HBM contract prices or new memory capacity would be the first signs of easing.

Capex guidance shows whether demand is still rising: Each of the last three revisions from Microsoft, Meta and Alphabet rose. A cut would be the first sign of softer demand. Until then, large buyers keep competing for the same supply.

Power approvals show how fast new data centres switch on: A data centre cannot run without a grid connection, and approvals take years. News of faster approvals or newly powered sites would add real supply.

Conclusion

GPU prices will stay uneven while memory and power stay scarce. Our view is that no single signal will end the shortage. The bottleneck has moved from GPUs to memory to power, and it is likely to move again.

We also see a split in how teams feel each spike. Teams that buy more GPU than the model needs pay for it every hour. Teams that match the GPU to the model, pick a billing plan that fits the load, and track cost per task feel spikes least.

For finance teams, the practical step is to budget a range, not a single rate. Our own rates will move with memory and supply costs too. That is why we publish GPU rates in INR on our pricing page, with data in India and no currency swings in the bill.