GPU Cloud for LLM Inference: Pricing and GPU Choice

◆

Where inference capacity earns its keep

What serving demands, workload by workload.

/01

Production model serving

Model serving for user-facing traffic where a failed request costs a customer. Single-node placement wherever the model fits, with headroom for peak concurrency.

vLLM · model serving · low latency

/02

Batch inference

Score large datasets where throughput matters and interruption does not. Cheaper and previous-generation silicon frequently wins here on cost per token.

Batch inference · offline scoring

/03

Long-context serving

Serve 128K context and beyond, including RAG pipelines, where the KV cache rather than the weights sets your memory floor.

128K+ context · KV cache · RAG

/04

Mixture-of-experts serving

Run mixture-of-experts models where total parameters far exceed active parameters, so memory capacity matters more than raw compute.

MoE · NVLink · tensor parallel

◆ HOW IT WORKS

A matchmaker, not a marketplace.

We connect you to our vetted partners. You contract directly with the operator running your nodes.

STEP 01/4

Tell us the requirement

GPU model, count, placement and timeline. Add workload detail if you have it.

STEP 02/4

We match capacity

We find vetted partners with capacity that fits, in the jurisdiction you need.

STEP 03/4

Quotes in 24 hours

Real quotes from partners who hold the capacity, not listings that may not exist.

STEP 04/4

Contract and provision

You contract directly with the operator. We smooth the provisioning process.

◆ FAQ

Frequently Asked Questions

Which GPU is best for LLM inference?

Memory capacity and bandwidth, in that order. If your model and KV cache fit in one GPU's memory you avoid cross-GPU communication entirely, which is the single biggest throughput gain available. H100 at 80 GB, H200 at 141 GB and B200 at 192 GB each move that threshold, and B300, GB300 and Vera Rubin move it further. All of those generations are available through the network.

How much does GPU cloud for LLM inference cost?

Work in cost per million tokens, not cost per GPU-hour. As of 31 August 2026 the median on-demand H100 rate was $3.33 per GPU-hour across 40 providers, B200 averaged $7.63 and H200 sat around $4.40 to $4.55. Newer generations are quoted per enquiry. A B200 at double the hourly rate is cheaper per token if it delivers more than double the throughput for your model, which it often does on long context and FP4-quantised serving.

What is the KV cache and why does it decide my GPU choice?

The KV cache holds attention state for every token in the context window, and it grows with context length and batch size. At long context it frequently consumes more memory than the model weights, which is why a 70B model that looks like it fits in 80 GB stalls in production. Sizing capacity without accounting for KV cache is the most common reason a deployment underperforms its benchmark.

Does vLLM, TensorRT-LLM or TGI change what hardware I need?

vLLM is the most widely deployed, with paged attention and continuous batching that raise throughput substantially over naive serving. TensorRT-LLM extracts more from NVIDIA hardware if you are willing to compile per model. Text Generation Inference sits between them for ease of deployment. All three run on any generation we place, so the serving stack rarely constrains the hardware choice.

What capacity structure suits inference?

Inference is steady-state load, so dedicated capacity on a commitment term generally fits it better than anything short-lived: you are paying for hours you actually use, and there is no eviction risk on user-facing traffic. Commitment terms and configurations vary by operator.

Where should inference capacity be placed?

Placement decides latency for your users and jurisdiction for their prompts. Prompts and responses are often personal data, so inference is the workload where residency rules bite hardest, more so than training. Tell us the jurisdiction you need and we return rates for capacity there, contracted directly with the operator.

◆ GET A QUOTE

Request wholesale rates

in under 24 hours.

Tell us the essentials. We'll line up real quotes from our vetted wholesale partners, and you contract directly with the operator.

◆Quotes in under 24 hours

◆Direct contact with operators

◆Vetted partners, matched to your requirement

◆20+ vetted providers · 10 regions

Thank you! Your submission has been received!

Oops! Something went wrong while submitting the form.

Thank you! Your submission has been received!

Oops! Something went wrong while submitting the form.

GPU cloud
for LLM inference
, at
wholesale price.

What inference capacity costs

What serving demands, workload by workload.

Production model serving

Batch inference

Long-context serving

Mixture-of-experts serving

Vetted GPU partners worldwide, sized for serving.

A matchmaker, not a marketplace.

Tell us the requirement

We match capacity

Quotes in 24 hours

Contract and provision

Frequently Asked Questions

GPU cloudfor LLM inference, atwholesale price.

What inference capacity costs

What serving demands, workload by workload.

Production model serving

Batch inference

Long-context serving

Mixture-of-experts serving

Vetted GPU partners worldwide, sized for serving.

A matchmaker, not a marketplace.

Tell us the requirement

We match capacity

Quotes in 24 hours

Contract and provision

Frequently Asked Questions

GPU cloud
for LLM inference
, at
wholesale price.