API model string
meta-llama/Llama-3.1-70B-Instruct (most common hosted; also 8B and 405B)Context window
128K tokens
Max output not published (host-capped, typically 4K–8K)
- Knowledge cutoff
- Dec 2023 (est.)
- Released
- Jul 2024
- Modalities
- text
Last verified July 10, 2026 — legacy model; Databricks retired 405B on Feb 15, 2026; data describes third-party host access
Rate limits by tier
Llama 3.1 has no first-party hosted API from Meta. Rate limits are set entirely by the inference host you choose. The 405B variant is actively losing support — Databricks retired it February 15, 2026.
| Tier | Requirements | RPM | TPM | RPD | Concurrent | Notes |
|---|---|---|---|---|---|---|
| Meta first-party | Waitlist only — not an active commercial endpoint | not published | not published | not published | not published | Meta's Llama API waitlist does not actively serve Llama 3.1 as a promoted SKU as of July 2026 |
| Groq (third-party) | Free tier available; paid tiers by usage | ~30 (free tier); higher on paid plans | not published (Groq-specific) | not published | not published | Llama 3.1 8B and 70B available on Groq; 405B support is host-dependent and thinning |
| Together AI (third-party) | Pay-as-you-go — credit balance required | not published | not published | not published | not published | 128K context honored; Llama 3.1 405B still listed as of July 2026; ~$0.88/MTok blended for 70B (per third-party trackers — confirm in dashboard) |
| DeepInfra (third-party) | Pay-as-you-go | not published | not published | not published | not published | ~$0.35 in / $0.40 out for 70B (per third-party trackers — confirm in dashboard); competitive per-token pricing |
| Amazon Bedrock (third-party) | AWS account; enable model access in Bedrock console | Per-account quota (request increase via Service Quotas) | not published | not published | not published | Batch inference ~50% off eligible throughput; Llama 3.1 70B and 405B available; quota increases take 1–2 days |
Swipe the table sideways to see every limit column.
- 1.All rate limits are set by the inference host, not Meta. Limits vary by host and are not publicly published in most cases — check your host's console.
- 2.405B availability is actively thinning: Databricks retired Meta-Llama-3.1-405B-Instruct on February 15, 2026. For 405B-class capability, evaluate Llama 4 Maverick (400B total, 17B active MoE, outperforms 405B dense on many benchmarks at lower cost).
- 3.128K is the model's native context window; all major hosts honor it for 70B/8B.
- 4.Llama 3.1 is open weights — no official Meta pricing exists. Prices are set by inference hosts and vary by model size and host.
- 5.Representative host prices (per third-party trackers — confirm in your console/dashboard before use): Together AI 70B ~$0.88/MTok blended; DeepInfra 70B ~$0.35 in / $0.40 out; Fireworks 8B ~$0.06 in / $0.06 out; 405B roughly 3–4× the 70B price.
- 6.Amazon Bedrock offers batch inference at ~50% off for eligible throughput. Other hosts' batch support varies.
- 7.No standard cross-host prompt caching. Cache system prompts at the application layer.
Limits verified against the Meta docs, July 10, 2026 — legacy model; Databricks retired 405B on Feb 15, 2026; data describes third-party host access.
meta-llama/Llama-3.1-70B-Instruct (most common hosted; also 8B and 405B) vs the alternatives
Llama 3.1 (legacy) compared to its active Llama 4 successors as of July 2026.
| Aspect | meta-llama/Llama-3.1-70B-Instruct (most common hosted; also 8B and 405B) | Llama 4 Scout | Llama 4 Maverick |
|---|---|---|---|
| Status | Legacy — host availability thinning | GA (live, broad support) | GA (live) |
| Context window | 128K | 10M native (128K–1M hosted) | 1M native (host-capped) |
| Cheapest input price | ~$0.35/MTok (70B, DeepInfra — verify) | $0.08/MTok (DeepInfra) | $0.15/MTok (DeepInfra) |
| Multimodal | Text only | Text + image | Text + image |
| 405B / max-params availability | 405B dense (Databricks retired Feb 2026; others thinning) | 109B total (16 experts active) | 400B total (128 experts active) |
| Host availability | Moderate (Databricks dropped 405B; thinning) | Broad — all major hosts | Broad — all major hosts |
| Batch discount | ~50% on Bedrock (verify) | Host-dependent (~50% on Bedrock) | Host-dependent (~50% on Bedrock) |
Swipe the table sideways to see every model.
Hitting a 429? The playbook
The exact errors you'll see
429 Too Many Requestsrate_limit_exceededRateLimitErrorThrottlingExceptionWhy it happens & how to fix it
Free Groq tier RPM exhausted on Llama 3.1 70B (~30 RPM limit)
Upgrade to Groq paid tier, or switch to DeepInfra pay-as-you-go where RPM limits are not publicly published but typically higher for paying customers.
Amazon Bedrock ThrottlingException — account quota limit reached
Request a quota increase in AWS Console: Service Quotas → Amazon Bedrock → select the specific Llama 3.1 model → request increase. Takes 1–2 business days.
405B host returning 429 or unavailable — availability thinning since Databricks retired it Feb 2026
Migrate to Llama 4 Maverick (400B total parameters, 17B active, comparable capability, broader host support). Update the model string and test prompts for any formatting differences.
Concurrent request limit on smaller hosts causing burst 429s
Spread requests across 2–3 hosts using round-robin routing. For example: route 50% to DeepInfra and 50% to Together AI, with health-check fallback logic.
Retry strategy
Use exponential backoff with jitter and honor the host's Retry-After header. All OpenAI-compatible hosts (Groq, Together, DeepInfra, Fireworks) and Bedrock return this header on 429/ThrottlingException. Formula: wait = min(2^attempt + random(0,1), 60). Cap retries at 5 attempts. OpenAI-compatible SDKs auto-retry on 429 if max_retries is set.
1import OpenAI from 'openai';23const client = new OpenAI({4 baseURL: 'https://api.deepinfra.com/v1/openai', // or api.groq.com/openai/v1, api.together.xyz/v15 apiKey: process.env.HOST_API_KEY,6 maxRetries: 0, // we handle retries manually7});89async function callWithRetry(10 prompt: string,11 maxRetries = 512): Promise<string> {13 for (let attempt = 0; attempt < maxRetries; attempt++) {14 try {15 const response = await client.chat.completions.create({16 model: 'meta-llama/Meta-Llama-3.1-70B-Instruct', // use host-specific alias17 messages: [{ role: 'user', content: prompt }],18 max_tokens: 2048,19 });20 return response.choices[0].message.content ?? '';21 } catch (err: any) {22 if (err?.status === 429) {23 const retryAfter = parseInt(err?.headers?.['retry-after'] ?? '0', 10);24 const jitter = Math.random();25 const wait = retryAfter > 026 ? retryAfter * 100027 : Math.min(Math.pow(2, attempt) + jitter, 60) * 1000;28 console.warn(`429 rate limit — attempt ${attempt + 1}/${maxRetries}, waiting ${Math.round(wait)}ms`);29 await new Promise(r => setTimeout(r, wait));30 } else {31 throw err;32 }33 }34 }35 throw new Error('Max retries exceeded after repeated 429 responses');36}How to raise your limits
The ladder from the starter tier to enterprise — what each rung takes, and what it unlocks.
Free host tier
ImmediateSign up on Groq (no credit card), Together AI, or DeepInfra (card required)
Unlocks: Low RPM (~30 on Groq free) on 8B and 70B models; 128K context on all major hosts
Pay-as-you-go
Immediate on paymentAdd credit balance of $10–$50 on Together AI, DeepInfra, or Fireworks
Unlocks: Higher RPM, full 128K context, 405B access where still available, competitive per-token rates
Amazon Bedrock production
1–2 business days for quota increase approvalEnable model access in Bedrock console → request throughput increase via AWS Service Quotas → Bedrock → select Llama 3.1 model
Unlocks: AWS ecosystem integration, batch inference at ~50% off, SLAs, enterprise compliance
Migrate to Llama 4
Hours to complete migrationUpdate model string to meta-llama/Llama-4-Scout-17B-16E-Instruct (or Maverick); adjust prompts for multimodal if needed
Unlocks: 10M native context (Scout) or 1M (Maverick), multimodal, lower cost, active host coverage, no thinning availability risk
Cut your token spend
Use 8B variant for simple tasks
~10× cheaper than 70B on most hosts; comparable quality on classification and structured extractionSwitch model string to meta-llama/Meta-Llama-3.1-8B-Instruct for routing, classification, or simple summarization tasks. Reserve 70B for complex reasoning.
Migrate to Llama 4 Scout for cost savings
~77% cheaper input vs 70B ($0.08 vs ~$0.35 on DeepInfra)Update model string to meta-llama/Llama-4-Scout-17B-16E-Instruct. Scout also offers 10M native context and multimodal support — upgrade with lower cost.
Use Amazon Bedrock batch inference
~50% off on eligible throughputSubmit non-real-time jobs (document processing, evals, bulk classification) as Bedrock batch jobs rather than synchronous API calls.
Cache system prompts at the application layer
Reduces redundant tokens — typically 20–40% token savings on repeat callsNo native cross-host prompt caching exists for Llama 3.1. Store and reuse system prompt tokens in your application; use a short prefix that remains constant across requests.
Keep context under 50K tokens per request
Improves host throughput and reduces per-request latencyChunk large documents to 30–50K tokens per request. This maximizes throughput on shared infrastructure and reduces the likelihood of hitting concurrent context limits.
Pin model version strings
Prevents unexpected behavior from host-side model version driftUse the full host-specific ID (e.g., meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo on Together AI) rather than generic aliases. Check the host's model list monthly.
Frequently asked questions
Is Llama 3.1 still available in 2026?
Llama 3.1 8B and 70B are still callable on major hosts including Groq, Together AI, DeepInfra, Fireworks, and Amazon Bedrock as of July 2026. However, the 405B variant was retired by Databricks on February 15, 2026, and availability of all variants is thinning. For new projects, Llama 4 Scout or Maverick are the recommended choices.
Is Llama 3.1 API free?
There is no first-party Meta-hosted Llama 3.1 API. Groq offers a free tier with approximately 30 RPM for Llama 3.1 8B and 70B. Paid hosts like Together AI, DeepInfra, and Fireworks charge per million tokens — rates vary from ~$0.06/MTok (8B on Fireworks) to ~$0.88/MTok blended (70B on Together, per third-party trackers; confirm in your dashboard).
How do I increase Llama 3.1 rate limits?
Rate limits for Llama 3.1 are set by your inference host, not Meta. To increase limits: upgrade from Groq free to a paid plan; add credit balance on Together AI, DeepInfra, or Fireworks; or request a quota increase on Amazon Bedrock via AWS Service Quotas (1–2 business days). For the highest and most stable limits, migrate to Llama 4 Scout or Maverick which have broader host support.
What happened to Llama 3.1 405B?
Databricks retired Meta-Llama-3.1-405B-Instruct on February 15, 2026. Some other hosts still list it, but availability is thinning. If you need 405B-class reasoning capability, Llama 4 Maverick (400B total parameters, 17B active via MoE architecture) is the recommended replacement — it outperforms the dense 405B on many benchmarks at lower per-token cost.
Llama 3.1 vs Llama 4 Scout: which should I use?
Llama 4 Scout for almost all new projects. It costs ~77% less on input ($0.08 vs ~$0.35 per MTok on DeepInfra), supports a 10M native context window (vs 128K for 3.1), and adds multimodal (text + image) support. Llama 3.1 has a slight edge only in existing integrations where migration cost is not justified yet.
Does Llama 3.1 support caching?
No native cross-host prompt caching exists for Llama 3.1. Amazon Bedrock does not offer Llama-specific prompt caching. You can implement application-layer caching by storing and reusing system prompt tokens in your backend, typically reducing repeated token spend by 20–40%.
What does a ThrottlingException mean on Amazon Bedrock for Llama 3.1?
ThrottlingException on Bedrock means your account has reached its per-model throughput quota. To fix it: go to AWS Console → Service Quotas → Amazon Bedrock → find the Llama 3.1 model → request a quota increase. This typically takes 1–2 business days. In the meantime, implement exponential backoff and honor the Retry-After header. If you need help sizing your Bedrock quota correctly, RapidDev offers free scoping calls at rapidevelopers.com/contact.
We build AI apps that don't hit rate limits
- Retry, backoff & caching built in
- Multi-provider fallback routing
- Fixed price, you own the code
30-min call. No commitment.