Feature spec
IntermediateCategory
communication-social
Build with AI
3–6 hours with V0 or Lovable
Custom build
3–7 days custom dev
Running cost
$5–15/mo at 100 users · $50–150/mo at 1K users
Works on
Everything it takes to ship an AI Chatbot — parts, prompts, and real costs.
An AI chatbot needs four pieces: a streaming LLM API route (Next.js Server Route calling Anthropic or OpenAI), the Vercel AI SDK useChat() hook rendering partial tokens as they arrive, conversation history stored in Supabase so context persists across page reloads, and Upstash Redis rate limiting that blocks API abuse before it reaches your LLM bill. With V0 or Lovable you can ship a working streaming chatbot in 3–6 hours. Cost at 100 users: $5–15/month in LLM tokens — the dominant cost at every scale.
What an AI Chatbot Feature Actually Is
An AI chatbot is a conversational interface that sends user messages to a large language model (Claude, GPT-4, or similar) and streams the response back token-by-token so users see text appearing in real time instead of waiting for a full reply. The visible product is the chat UI — message bubbles, a typing indicator, a fixed-bottom input bar. The invisible architecture is what separates a working chatbot from one that burns your API budget: the system prompt that defines the bot's personality, the conversation history array sent with every message to give the model context, a server-side API route keeping your API key out of the browser, and rate limiting that prevents one user from draining a month of budget in an afternoon.
What users consider table stakes in 2026
- Streaming token-by-token response display — users see text appearing in real time, not a spinner followed by a wall of text
- Conversation history preserved across page navigations and browser refreshes, loaded from Supabase on mount
- Typing indicator (three animated dots) while the AI is generating and before the first token arrives
- Stop button that cancels the active generation mid-stream — critical for users who asked the wrong question
- Message timestamps and a clear conversation history showing who said what
- Mobile-friendly fixed-bottom input bar that stays above the keyboard on iOS with env(safe-area-inset-bottom)
Anatomy of the Feature
Six components. The LLM API route and the rate limiter are where most first builds skip steps that matter — API key exposure and runaway costs are the two failure modes that kill chatbot features before they ship.
Chat UI component
UIA client component with a shadcn/ui ScrollArea rendering a list of MessageBubble components (user messages right-aligned, AI messages left-aligned). Auto-scrolls to the latest message using a ref on the scroll container. Framer Motion fade-in animation on each new message. Empty state shows 3–5 suggested starter prompts the user can click to pre-fill the input.
Note: Use virtualization (TanStack Virtual) for conversations exceeding 100 messages to keep scroll performance smooth on long chats.
Streaming response handler
UIThe Vercel AI SDK useChat() hook manages the message array, the input field, and the streaming state. It sends the full conversation history with each request and appends incoming tokens to the last AI message as they arrive. The hook exposes isLoading (typing indicator), stop() (cancel generation), and messages (the conversation array).
Note: Import useChat from 'ai/react' (Vercel AI SDK). It handles the ReadableStream from the API route automatically — no manual SSE parsing needed.
LLM API route
BackendA Next.js App Router Route Handler at app/api/chat/route.ts. Receives the messages array from the client, loads the system prompt from Supabase app_config, calls Anthropic SDK streamText() or OpenAI SDK with stream:true, and returns a ReadableStream. Runs server-side so ANTHROPIC_API_KEY never reaches the client bundle.
Note: Use the Vercel AI SDK streamText() wrapper which handles both Anthropic and OpenAI with the same interface — easier to swap models later. Never call LLM APIs directly from a client component.
Conversation memory
DataA Supabase conversations table storing each conversation's messages as a JSONB array. Loaded on component mount and passed to useChat() as initialMessages. After each AI response completes, the updated messages array is upserted back to Supabase. A conversations list sidebar shows the user's previous chats by title (auto-generated from the first user message).
Note: Implement a sliding window when conversation history grows large: only send the last N messages that fit within approximately 80K tokens. Summarize older messages using the LLM itself and store the summary as a compressed 'context' field.
System prompt configuration
DataA Supabase app_config table with a chatbot_system_prompt text column that the API route fetches on every request. This lets you update the chatbot's personality, instructions, and knowledge scope from an admin panel without redeploying. Cache the config in-process for 60 seconds to avoid a DB hit per message.
Note: Never hardcode the system prompt in the API route — admins need to adjust it to fix hallucinations and add product-specific context without engineering involvement.
Rate limiting
BackendUpstash Redis @upstash/ratelimit with a sliding window algorithm: 20 messages per authenticated user per hour. The rate limiter runs at the top of the API route before any LLM call. Returns a 429 response with a 'Rate limit exceeded' message the UI displays to the user. Unauthenticated requests are limited by IP address at a lower threshold (5 per hour).
Note: Upstash Redis is the correct solution for serverless rate limiting — in-memory Maps in serverless functions reset on every cold start, making them useless for rate limiting.
The data model
Two tables: conversations (stores message history as JSONB) and chat_usage (tracks token consumption per user per day for quota management). Run this in the Supabase SQL Editor.
1create table public.conversations (2 id uuid primary key default gen_random_uuid(),3 user_id uuid references auth.users(id) on delete cascade not null,4 title text not null default 'New conversation',5 messages jsonb not null default '[]',6 model text not null default 'claude-sonnet-4-5',7 created_at timestamptz not null default now(),8 updated_at timestamptz not null default now()9);1011create table public.chat_usage (12 id uuid primary key default gen_random_uuid(),13 user_id uuid references auth.users(id) on delete cascade not null,14 date date not null default current_date,15 message_count integer not null default 0,16 token_count integer not null default 0,17 unique (user_id, date)18);1920create table public.app_config (21 key text primary key,22 value text not null,23 updated_at timestamptz not null default now()24);2526-- Insert default system prompt27insert into public.app_config (key, value) values28 ('chatbot_system_prompt', 'You are a helpful assistant. Answer questions clearly and concisely. If you are unsure about something, say so.');2930-- Indexes31create index conversations_user_idx on public.conversations (user_id, updated_at desc);32create index chat_usage_user_date_idx on public.chat_usage (user_id, date desc);3334-- RLS35alter table public.conversations enable row level security;36alter table public.chat_usage enable row level security;37alter table public.app_config enable row level security;3839-- Users can only access their own conversations40create policy "Users can view own conversations"41 on public.conversations for select42 using (user_id = auth.uid());4344create policy "Users can insert own conversations"45 on public.conversations for insert46 with check (user_id = auth.uid());4748create policy "Users can update own conversations"49 on public.conversations for update50 using (user_id = auth.uid());5152create policy "Users can delete own conversations"53 on public.conversations for delete54 using (user_id = auth.uid());5556-- Chat usage: users see own, admins see all57create policy "Users can view own usage"58 on public.chat_usage for select59 using (user_id = auth.uid());6061-- app_config: readable by authenticated users (system prompt is not sensitive)62create policy "Authenticated users can read config"63 on public.app_config for select64 using (auth.uid() is not null);Heads up: The messages JSONB column stores the full conversation as an array of { role: 'user' | 'assistant', content: string } objects — the exact format Anthropic and OpenAI both expect. The chat_usage table with a unique (user_id, date) constraint lets you increment token counts with an upsert rather than separate insert/update logic.
Build it — pick your path
Each tool fits this feature differently. Switch tabs to compare fit, time, the exact steps, and a copy-paste prompt.
Best path — V0 generates the Vercel AI SDK useChat() hook with streaming out of the box, and Vercel deployment handles API routes as serverless functions without extra configuration.
Step by step
- 1Prompt V0 with the chatbot spec below — it generates the chat UI, the API route with streaming, and the conversation sidebar
- 2Add environment variables in V0's Vars panel: ANTHROPIC_API_KEY (server-side, no NEXT_PUBLIC_ prefix), NEXT_PUBLIC_SUPABASE_URL, NEXT_PUBLIC_SUPABASE_ANON_KEY, SUPABASE_SERVICE_ROLE_KEY, UPSTASH_REDIS_REST_URL, and UPSTASH_REDIS_REST_TOKEN
- 3Run the SQL schema from this page in the Supabase SQL Editor
- 4Create an Upstash Redis database at upstash.com (free tier: 10,000 commands/day) and copy the REST URL and token
- 5Deploy to Vercel — streaming works correctly only in the deployed environment, not in V0's preview sandbox
Build a streaming AI chatbot feature in Next.js App Router using the Vercel AI SDK and Anthropic. API route: app/api/chat/route.ts — import { streamText } from 'ai'; import { anthropic } from '@ai-sdk/anthropic'; export async function POST(req: Request) { const { messages, conversationId } = await req.json(); // Rate limit check using @upstash/ratelimit with Upstash Redis: 20 messages per user per hour, 5 per IP for unauthenticated; return 429 with json { error: 'Rate limit exceeded. Try again in X minutes.' } if exceeded. Load system prompt from Supabase app_config table (key = 'chatbot_system_prompt'). Call streamText({ model: anthropic('claude-sonnet-4-5'), system: systemPrompt, messages, maxTokens: 2048 }); return result.toDataStreamResponse(). After stream completes, upsert conversation to Supabase conversations table (user_id, messages array, updated_at). Chat UI component (client): import { useChat } from 'ai/react'. Chat layout: full-height flex column — top: conversation title + Clear button; middle: ScrollArea with MessageBubble components (user right/blue, AI left/gray), each with role icon and relative timestamp; fade-in animation per message using Framer Motion; auto-scroll to bottom ref on messages change; typing indicator (three animated dots) when isLoading and last message is user role. Bottom: fixed input bar — shadcn/ui Textarea (auto-resize, Enter to submit, Shift+Enter for newline), Send button, Stop button (calls stop() from useChat) shown during isLoading, padding-bottom: env(safe-area-inset-bottom) for iOS keyboard. Empty state: show 4 suggested starter prompts as clickable chips that call append() on click. On mount: fetch conversations list from Supabase (title, updated_at, id) ordered by updated_at DESC; load conversation messages from Supabase by conversationId and pass as initialMessages to useChat. On each AI response complete: auto-generate conversation title from first user message (take first 50 chars). Error handling: show 'I am having trouble responding, please try again' as an AI message bubble on API error. ANTHROPIC_API_KEY must stay server-side — never in a client component.Where this path bites
- Streaming does not work in V0's preview sandbox — deploy to Vercel to test the streaming experience
- Supabase setup is manual — V0 does not auto-provision the database; run the SQL schema in the Supabase SQL Editor
- Upstash Redis requires a separate free account — V0 does not provision it
Third-party services you'll need
Four services. LLM tokens are the dominant cost at every scale — budget for them first.
| Service | What it does | Free tier | Paid from |
|---|---|---|---|
| Anthropic API | LLM inference — claude-sonnet-4-5 is the recommended default; claude-haiku-4-5 for cost-sensitive high-volume use cases | $5 credit on signup | claude-sonnet-4-5: $3/$15 per million tokens in/out; claude-haiku-4-5: $0.25/$1.25 per million tokens |
| OpenAI API | Alternative LLM — gpt-4o-mini for cost efficiency, gpt-4o for highest capability; same streaming interface via Vercel AI SDK | $5 credit on signup | gpt-4o-mini: $0.15/$0.60 per million tokens; gpt-4o: $2.50/$10 per million tokens |
| Upstash Redis | Persistent rate limiting state across serverless function invocations — sliding window algorithm, 20 messages per user per hour | Free: 10,000 commands/day | Pay-per-use from approx $0.20/100,000 commands |
| Vercel AI SDK | Streaming abstraction for useChat() hook and streamText() server function — handles ReadableStream from Anthropic and OpenAI with a unified interface | Free, open source | $0 |
Swipe the table sideways to see pricing.
What it costs to run
Drag through the tiers to see how your monthly bill scales with users — no surprises later.
Estimated monthly running cost
LLM tokens dominate at every tier. Estimate: 500 tokens/message average, 10 messages/user/day, 100 users = 500K tokens/day. Claude Sonnet at $3/$15 per million in/out = approximately $8–15/mo. Supabase and Upstash free tiers cover infrastructure.
Estimates use verified 2026 pricing and assume typical usage per user. Your real bill depends on activity, storage, and third-party plans.
What breaks when AI tools build this
The failures people actually hit on their first build — the symptom, why it happens, and the exact fix.
Streaming breaks in Lovable preview — messages appear all at once
Symptom: Supabase Edge Function SSE streaming does not work reliably through Lovable's preview iframe proxy. The iframe buffers the SSE response and releases it all at once when the stream closes, making the chatbot appear non-streaming during development. This is a preview-only issue — streaming works correctly in production.
Fix: Test streaming exclusively on the published URL (Publish → Update in Lovable). Do not judge streaming behavior from the preview pane. If streaming still doesn't work on the live URL, verify the Edge Function is returning a ReadableStream with the correct Content-Type: text/event-stream header.
Conversation history exceeds the LLM context window and returns a 400 error
Symptom: The useChat() hook sends all stored messages with every request. After 20–30 exchanges, the total token count of the messages array plus the system prompt approaches or exceeds the model's context window. Claude Sonnet has a 200K token context window, but conversations with long messages can hit this surprisingly fast. The API returns a 400 error that the UI usually renders as a generic failure.
Fix: Implement a sliding window in the API route: count the tokens of the messages array (roughly 4 characters per token) and trim from the oldest messages until the total fits within 80K tokens (leaving headroom for the system prompt and response). Optionally, summarize the trimmed portion with a quick Haiku call and inject the summary as a system message.
V0 generates localStorage conversation persistence that crashes on SSR
Symptom: V0 sometimes stores conversation history in localStorage using code at module scope — outside a useEffect. In Next.js App Router, module-scope code runs server-side during the initial render. localStorage is undefined server-side, throwing 'localStorage is not defined' and crashing the page hydration.
Fix: Wrap all localStorage access in a useEffect (client-only) or use Supabase for conversation storage (server-safe). The prompt above instructs V0 to load conversation history from Supabase on mount — verify the generated code does not access localStorage before the component mounts.
ANTHROPIC_API_KEY exposed in the client bundle
Symptom: V0 occasionally generates the Anthropic API call directly inside a React client component using useEffect — the API key is passed as a header from the browser. Any user who opens the browser Network tab can read the key. This leads to immediate key exposure if the app is deployed publicly.
Fix: All LLM API calls must go through a Next.js API route (app/api/chat/route.ts) or Supabase Edge Function where the key is only accessible as a server environment variable. In V0: if you see ANTHROPIC_API_KEY referenced in a file with 'use client' at the top, that is the bug. Move the API call to a server route.
In-memory rate limiting resets on every serverless cold start
Symptom: A common mistake: implementing rate limiting with a Map or object stored in the API route module scope. This works in development (single long-running Node process) but completely fails in production serverless functions — every function invocation potentially gets a fresh cold start, resetting the in-memory counter. Users can exceed rate limits by just waiting for a new cold start.
Fix: Always use Upstash Redis for rate limiting in serverless environments. Upstash persists the sliding window counter across all function invocations. The @upstash/ratelimit package handles the sliding window algorithm with a single import and three lines of code.
Best practices
Keep ANTHROPIC_API_KEY and OPENAI_API_KEY strictly server-side — never in a client component, never with a NEXT_PUBLIC_ prefix
Send the full conversation history with every message, but implement a sliding window that trims the oldest messages when the total exceeds 80K tokens
Load the system prompt from a Supabase config table, not from hardcoded strings — product teams need to adjust chatbot behavior without engineering deploys
Use Upstash Redis for rate limiting — never in-memory rate limiting in serverless functions
Implement the Stop button from day one — useChat exposes a stop() function; users who ask a long question and immediately want to rephrase it will expect to be able to cancel
Generate conversation titles automatically from the first user message (first 50 characters) — users can't remember which of their 20 'New conversation' chats has the information they want
Handle the 429 rate limit response gracefully in the UI — show the rate limit reset time, not a generic error message
When You Need Custom Development
V0 and Lovable ship a solid single-model streaming chatbot. These requirements need custom development:
- Multi-model routing — routing simple queries to Haiku and complex reasoning to Sonnet based on automatic query classification, reducing LLM costs by 60–70% at scale
- RAG pipeline with pgvector — embedding your product documentation, knowledge base, or user data and injecting relevant chunks into the system prompt for grounded, accurate responses
- Customer support chatbot with human handoff — detecting when the AI reaches its confidence threshold and routing the conversation to a human agent in Intercom or Zendesk with full conversation context transferred
- Per-conversation billing with Stripe metered usage — tracking token counts per user per conversation and charging overages at the end of each billing period
RapidDev builds these features for production
Real auth, database, and payments — fixed price, delivered in weeks. You own 100% of the code.
Frequently asked questions
How do I add conversation history to my AI chatbot?
Store each conversation's messages as a JSONB array in a Supabase conversations table. On the useChat() hook, pass the loaded messages as initialMessages — the hook sends the full array with every new message, giving the model context. After each AI response completes, upsert the updated messages array back to Supabase. The SQL schema above includes the full conversations table with RLS policies.
What's the difference between streaming and non-streaming chatbot responses?
Non-streaming waits for the LLM to finish generating the entire response before sending it — users stare at a spinner for 5–30 seconds. Streaming sends each token as it's generated — users see text appearing word by word within 200ms of sending their message. Streaming feels dramatically more responsive. The Vercel AI SDK's useChat() hook handles streaming automatically with ReadableStream.
How do I prevent users from abusing my chatbot API?
Upstash Redis + @upstash/ratelimit is the standard solution for serverless rate limiting. Configure a sliding window: 20 messages per authenticated user per hour, 5 per IP for unauthenticated users. The rate limit check runs in the API route before any LLM call — a 429 response costs $0 in LLM tokens. Set budget alerts in both Anthropic Console and OpenAI Dashboard as a secondary protection.
How much does an AI chatbot cost per month?
LLM tokens are the dominant cost. At 10 messages per user per day with 500 average tokens per message: 100 users = approximately $10–15/mo on Claude Sonnet; 1,000 users = $100–150/mo; 10,000 users = $1,000–1,500/mo. Switching to Claude Haiku for simpler queries reduces cost by approximately 85%. Infrastructure (Supabase, Vercel, Upstash) adds $0–50/mo depending on scale.
Can I use Claude and GPT-4 in the same chatbot?
Yes — the Vercel AI SDK provides a unified interface for both. In the API route, you can call anthropic('claude-sonnet-4-5') for some requests and openai('gpt-4o') for others. A common pattern: use Claude for nuanced, context-rich conversations and GPT-4o-mini for simple factual lookups. Pass the selected model name as a field in the request body and store it in the conversations.model column so each conversation remembers which model was used.
How do I add a system prompt that defines my chatbot's personality?
Store the system prompt in a Supabase app_config table with a key of 'chatbot_system_prompt'. In the API route, fetch it with `supabase.from('app_config').select('value').eq('key', 'chatbot_system_prompt').single()` and pass it as the system parameter to streamText(). Cache the result for 60 seconds in-process to avoid a DB hit per message. Build an admin UI that lets non-technical team members update the system prompt without redeploying.
How do I store chat history in Supabase?
After each AI response completes (use the onFinish callback in useChat), call supabase.from('conversations').upsert({ id: conversationId, user_id: userId, messages: updatedMessages, updated_at: new Date().toISOString() }). Load the history on mount: fetch the conversation row by id and pass messages as initialMessages to useChat(). The SQL schema above includes the full conversations table with correct RLS policies.
How do I stop a chatbot response mid-generation?
The Vercel AI SDK useChat() hook exposes a stop() function that aborts the active ReadableStream. Wire it to a Stop button that appears only when isLoading is true. When stop() is called, the partial AI response so far is preserved in the messages array as the AI's reply — the conversation continues from that point. Do not show the Stop button after the stream completes — toggle it with the isLoading state.
Need this feature production-ready?
RapidDev builds an ai chatbot into real apps — auth, database, payments — at $13K–$25K.
Book a free consultation30-min call. No commitment.