API model string
gemma-2 (no current first-party hosted API SKU; open weights on Hugging Face)Context window
8K tokens (2B, 9B, 27B variants)
Max output not published
- Knowledge cutoff
- not published
- Released
- 2024
- Modalities
- text in, text out (2B, 9B, 27B sizes)
Last verified July 10, 2026
Rate limits by tier
No meaningful first-party hosted API SKU for Gemma 2 exists in Google's current 2026 lineup. The primary access path is self-hosting open weights. Limits below describe what is published for each access path, plus Gemma 3 as the recommended hosted alternative.
| Tier | Requirements | RPM | TPM | RPD | Concurrent | Notes |
|---|---|---|---|---|---|---|
| First-party hosted API (Google AI) | n/a — no current promoted SKU | not published | not published | not published | not published | No meaningful first-party hosted API SKU for Gemma 2 is promoted in Google's 2026 AI lineup. Attempting to call a Gemma 2 string via the standard API may return model-not-found errors. |
| Self-host (primary path) | Gemma License (commercial use allowed); GPU hardware | unlimited (hardware-bound) | unlimited | unlimited | hardware-bound | Weights: google/gemma-2-2b-it, google/gemma-2-9b-it, google/gemma-2-27b-it on Hugging Face. 2B runs on consumer GPU (RTX 3070+); 27B requires server GPU. Zero per-token cost. |
| Hugging Face Inference API | Hugging Face account (free or Pro) | not published | not published | not published | not published | Available for inference; rate-limited on HF free tier. HF Pro or Dedicated Endpoints required for production throughput. Billed by compute-hour on Dedicated Endpoints. |
| Google AI Studio (legacy) | Google account | not published | not published | not published | not published | Gemma 2 may appear in AI Studio for legacy use but is not a promoted current model. Limits not published. Consider Gemma 3 for any new or continuing use in AI Studio. |
Swipe the table sideways to see every limit column.
- 1.Gemma 2 has been superseded by Gemma 3 (March 2025) and Gemma 4 (2026). The 8K context window is a significant limitation compared to Gemma 3's 131K.
- 2.No meaningful first-party commercial hosting exists for Gemma 2 in 2026. Self-host via Hugging Face or Ollama is the primary path.
- 3.Open weights remain available under the Gemma License (commercial use permitted). Weights are not being sunset — only the hosted API SKU is absent.
Limits verified against the Google docs, July 10, 2026.
Token pricing
What you pay per million tokens (USD). Input and output are billed separately.
Input
$0.04
per 1M tokens
Output
$0.08
per 1M tokens
- Gemma 2 first-party hosted API pricing is not published (no current SKU). The numbers above are for Gemma 3 4B hosted via Google AI — the recommended migration target.
- Gemma 2 self-host: zero per-token cost. You pay for compute (GPU instance hours) only.
- Hugging Face Dedicated Endpoints for Gemma 2: billed by compute hour; variable by instance type. See huggingface.co/docs/inference-endpoints for current rates.
- Gemma 3 hosted batch: 50% discount available on Google AI PAYG.
Self-hosted Gemma 2 9B (legacy path)
~$1,080/month (compute only; no per-token fee)
per month
Assumptions
Inference on 1× A10 spot instance (~$1.50/hr), ~100 req/hr, 720 hours/month at 100% utilization
720 hours × $1.50/hr = $1,080
Gemma 3 4B hosted — recommended upgrade (small scale)
$0.28
per month
Assumptions
5M input tokens / 1M output tokens per month via Google AI
5M × $0.040/1M + 1M × $0.080/1M = $0.20 + $0.08
Gemma 3 4B hosted — recommended upgrade (scale)
$22.40
per month
Assumptions
400M input tokens / 80M output tokens per month
400M × $0.040/1M + 80M × $0.080/1M = $16 + $6.40
Run your own numbers
Drag your real monthly token volumes and watch the bill update live.
Estimated gemma-2 (no current first-party hosted API SKU; open weights on Hugging Face) spend
$1.60/mo
Input: $1.20
Output: $0.40
30M in × $0.040 + 5M out × $0.080 = $1.60
Estimate for comparison only. Real bills vary with request shape, long-context surcharges, and thinking-token usage.
gemma-2 (no current first-party hosted API SKU; open weights on Hugging Face) vs the alternatives
Gemma 2 is compared against its direct successor and current open-weight alternatives. On every meaningful axis, newer models are the better choice for 2026 projects.
| Aspect | gemma-2 (no current first-party hosted API SKU; open weights on Hugging Face) | Gemma 3 | Phi-3 | Llama 4 Scout |
|---|---|---|---|---|
| Context window | 8K tokens (Gemma 2) | 131K tokens (Gemma 3) | 128K tokens (Phi-4-mini) | 128K–1M tokens (Llama 4 Scout) |
| First-party hosted API price | not published (no current SKU) | $0.040/$0.080 per MTok (Gemma 3 4B) | $0.07/$0.23 per MTok (Phi-4-mini) | $0.08–$0.11/$0.30–$0.34 per MTok (Llama 4 Scout/Groq) |
| Self-host availability | Yes (Gemma License, commercial allowed) | Yes (Gemma License) | Yes (MIT) | Yes (Llama 4 license) |
| Multimodal (vision) | No — text only | Yes (Gemma 3 vision) | Yes (Phi-4) | Yes (Llama 4 Scout) |
| Model generation | 2024 (legacy) | 2025 — Gemma 3 (current) | 2024–2025 — Phi-4 (current) | 2025 — Llama 4 (current) |
| Hosted batch discount | n/a (no hosted SKU) | 50% (Gemma 3 hosted) | Varies (Azure/third-party) | Varies (host-dependent) |
Swipe the table sideways to see every model.
Hitting a 429? The playbook
The exact errors you'll see
429 Too Many RequestsRESOURCE_EXHAUSTEDQuota exceededmodels/gemma-2 is not found for API version v1betaWhy it happens & how to fix it
Attempting to call Gemma 2 via the current Google AI hosted API
Gemma 2 has no promoted hosted API SKU. Update your model string to gemma-3-4b-it (or 12b/27b) to access the first-party hosted API. The Gemma 2 string may return model-not-found.
Hugging Face Inference API rate limit on free tier
Upgrade to Hugging Face Pro, or deploy a Dedicated Endpoint (huggingface.co/docs/inference-endpoints) for production throughput. Dedicated Endpoints provide a private, SLA-backed inference server billed by compute hour.
Self-hosted OOM (out of memory) error
Use a smaller model size (2B instead of 9B or 27B), or apply int4 quantization to reduce GPU memory by 4×. Ollama supports quantized Gemma 2 out of the box: ollama pull gemma2:9b.
HF free-tier concurrency limit hit
Deploy a dedicated inference server using TGI (Text Generation Inference), vLLM, or Ollama on your own GPU instance. This removes rate limits and provides continuous batching for higher throughput.
Retry strategy
For Hugging Face Inference API: use exponential backoff with the Retry-After header. For self-hosted inference (TGI, vLLM, Ollama): retry logic lives in your client code — TGI and vLLM return standard HTTP 429 when overloaded. Start at 1s, double per retry, cap at 30s. Add ±20% jitter.
1// retry.ts — Hugging Face Inference API for Gemma 2 (self-host migration path)2// For new projects, prefer Gemma 3 on Google AI: model = 'gemma-3-4b-it'3const HF_TOKEN = process.env.HF_TOKEN!;4const MODEL = "google/gemma-2-9b-it";5const HF_URL = `https://api-inference.huggingface.co/models/${MODEL}`;67async function generateWithRetry(8 prompt: string,9 maxRetries = 510): Promise<string> {11 let delay = 1000;12 for (let attempt = 0; attempt <= maxRetries; attempt++) {13 const res = await fetch(HF_URL, {14 method: "POST",15 headers: {16 Authorization: `Bearer ${HF_TOKEN}`,17 "Content-Type": "application/json"18 },19 body: JSON.stringify({ inputs: prompt })20 });2122 if (res.ok) {23 const data = await res.json();24 return Array.isArray(data) ? data[0].generated_text : data.generated_text;25 }2627 if (res.status === 429 && attempt < maxRetries) {28 const retryAfter = res.headers.get("Retry-After");29 const wait = retryAfter30 ? parseInt(retryAfter, 10) * 100031 : delay * (1 + Math.random() * 0.4 - 0.2);32 console.warn(`429 — retrying in ${Math.round(wait / 1000)}s (attempt ${attempt + 1})`);33 await new Promise((r) => setTimeout(r, wait));34 delay = Math.min(delay * 2, 30000);35 continue;36 }3738 throw new Error(`HF API error ${res.status}: ${await res.text()}`);39 }40 throw new Error("Max retries exceeded");41}How to raise your limits
The ladder from the starter tier to enterprise — what each rung takes, and what it unlocks.
Local / dev
MinutesRun 'ollama pull gemma2:9b' (or gemma2:2b for smaller hardware) and call the local endpoint at localhost:11434.
Unlocks: Local inference at zero cost. No rate limits. Requires an NVIDIA or Apple Silicon GPU.
Hugging Face free tier
ImmediatePOST to https://api-inference.huggingface.co/models/google/gemma-2-9b-it with Authorization: Bearer {HF_TOKEN}.
Unlocks: Limited free inference suitable for testing. Rate-limited and not suitable for production.
Hugging Face Dedicated Endpoint
MinutesGo to huggingface.co/docs/inference-endpoints → create a new endpoint → select google/gemma-2-9b-it → choose GPU instance → deploy.
Unlocks: Production SLA, autoscale, custom GPU selection. Billed by compute hour.
Self-hosted cloud VM
HoursDeploy TGI or vLLM on a GPU VM (GCP, AWS, Lambda Labs, CoreWeave). Load google/gemma-2-9b-it weights from Hugging Face. Use int4 quantization to reduce GPU memory requirements.
Unlocks: Full hardware control, highest throughput via continuous batching, no per-token vendor fee.
Migrate to Gemma 3 (recommended)
Minutes for the code changeUpdate model ID from gemma-2-9b-it to gemma-3-4b-it (or 12b/27b) and switch to Google AI hosted API. Enable billing on your Google Cloud project.
Unlocks: 131K context (vs 8K), multimodal vision, first-party hosted API at $0.040/$0.080 per MTok, 50% batch discount.
Cut your token spend
Migrate to Gemma 3
131K context vs 8K — 16× more context capacity; multimodal supportThe single highest-impact action for any Gemma 2 project. Gemma 3 (gemma-3-4b-it) runs on the same self-host infrastructure, has a first-party hosted API at $0.040/MTok, and adds vision support. Gemma 4 is even newer (2026) — evaluate both before committing to Gemma 3 long-term.
Use int4 quantization for self-hosted Gemma 2
4× GPU memory reduction; enables smaller / cheaper hardwareOllama includes quantized Gemma 2 variants by default. For vLLM: specify --quantization awq or --quantization gptq. Quantized 9B models can run on a single 16GB GPU vs 40GB+ for fp16.
Local inference via Ollama for zero-cost dev
Zero token cost; no rate limitsRun 'ollama run gemma2' on a laptop or dev machine. Suitable for local testing, prompt iteration, and fine-tune evaluation. No API key, no billing, no rate limits — fully offline.
Continuous batching on self-hosted vLLM
3–8× higher throughput vs sequential requestsvLLM's continuous batching processes incoming requests without stopping for batch boundaries. Start the server with 'vllm serve google/gemma-2-9b-it --enable-chunked-prefill'. Use async client calls to keep the request queue full.
Consider Gemma 2 only for existing fine-tuned checkpoints
Avoids re-training costIf you have existing Gemma 2 fine-tuned weights that encode domain knowledge, migrating to Gemma 3 requires re-training on the new architecture. In that specific case, continuing to use self-hosted Gemma 2 may be justified — but evaluate the 8K context ceiling against your use case.
Frequently asked questions
Does Gemma 2 have a hosted API?
No. No meaningful first-party hosted API SKU for Gemma 2 is promoted in Google's 2026 lineup. Attempting to call a Gemma 2 model string via the standard Google AI API may return model-not-found errors. The primary access path is self-hosting open weights via Hugging Face or Ollama under the Gemma License.
What is Gemma 2's context window?
Gemma 2 has an 8K token context window across all sizes (2B, 9B, 27B). This is a significant limitation compared to Gemma 3's 131K context window. If your use case involves long documents, multi-turn conversations, or large code files, Gemma 2 is unsuitable for most 2026 tasks.
Should I migrate from Gemma 2 to Gemma 3?
Yes, for almost all use cases. Gemma 3 offers 131K context (vs 8K), multimodal vision support, a first-party hosted API at $0.040/$0.080 per MTok, and a 50% batch discount. The only reason to stay on Gemma 2 is if you have fine-tuned checkpoints on the Gemma 2 architecture that you cannot afford to retrain. Gemma 4 (2026) is also available — evaluate it for new projects.
How do I run Gemma 2 locally?
The easiest path is Ollama: run 'ollama pull gemma2:9b' (or gemma2:2b for smaller hardware) and call localhost:11434. For production self-hosting, use vLLM or TGI with the Hugging Face weights (google/gemma-2-2b-it, google/gemma-2-9b-it, google/gemma-2-27b-it). Apply int4 quantization to reduce GPU memory requirements by 4×.
Is Gemma 2 free to use commercially?
Yes. The open weights are available under the Gemma License, which allows commercial use. You self-host the weights and pay only for your compute (no per-token fees). Check the full Gemma License terms at ai.google.dev/gemma/terms for any commercial restrictions.
Gemma 2 vs Gemma 3 — what changed?
Context window: 8K (Gemma 2) vs 131K (Gemma 3). Multimodal: Gemma 2 is text-only; Gemma 3 adds vision input. Hosted API: Gemma 2 has no current promoted hosted SKU; Gemma 3 4B is available at $0.040/$0.080 per MTok via Google AI. Model generation: Gemma 2 is 2024; Gemma 3 is 2025 (current). For new projects, Gemma 3 is the unambiguous choice.
Can RapidDev help me evaluate Gemma 2 vs Gemma 3 for my use case?
Yes. RapidDev can audit your current Gemma 2 setup, assess whether your fine-tuned checkpoints are worth preserving, and design a migration path to Gemma 3 with caching and batching optimised for your workload. Book a free scoping call at rapidevelopers.com/contact.
We build AI apps that don't hit rate limits
- Retry, backoff & caching built in
- Multi-provider fallback routing
- Fixed price, you own the code
30-min call. No commitment.