# How to Add an AI Chatbot to Your App (Copy-Paste Prompts Included)

- Tool: App Features
- Last updated: July 2026

## TL;DR

An AI chatbot needs four pieces: a streaming LLM API route (Next.js Server Route calling Anthropic or OpenAI), the Vercel AI SDK useChat() hook rendering partial tokens as they arrive, conversation history stored in Supabase so context persists across page reloads, and Upstash Redis rate limiting that blocks API abuse before it reaches your LLM bill. With V0 or Lovable you can ship a working streaming chatbot in 3–6 hours. Cost at 100 users: $5–15/month in LLM tokens — the dominant cost at every scale.

## What an AI Chatbot Feature Actually Is

An AI chatbot is a conversational interface that sends user messages to a large language model (Claude, GPT-4, or similar) and streams the response back token-by-token so users see text appearing in real time instead of waiting for a full reply. The visible product is the chat UI — message bubbles, a typing indicator, a fixed-bottom input bar. The invisible architecture is what separates a working chatbot from one that burns your API budget: the system prompt that defines the bot's personality, the conversation history array sent with every message to give the model context, a server-side API route keeping your API key out of the browser, and rate limiting that prevents one user from draining a month of budget in an afternoon.

## Anatomy of the Feature

Six components. The LLM API route and the rate limiter are where most first builds skip steps that matter — API key exposure and runaway costs are the two failure modes that kill chatbot features before they ship.

- **Chat UI component** (ui): A client component with a shadcn/ui ScrollArea rendering a list of MessageBubble components (user messages right-aligned, AI messages left-aligned). Auto-scrolls to the latest message using a ref on the scroll container. Framer Motion fade-in animation on each new message. Empty state shows 3–5 suggested starter prompts the user can click to pre-fill the input.
- **Streaming response handler** (ui): The Vercel AI SDK useChat() hook manages the message array, the input field, and the streaming state. It sends the full conversation history with each request and appends incoming tokens to the last AI message as they arrive. The hook exposes isLoading (typing indicator), stop() (cancel generation), and messages (the conversation array).
- **LLM API route** (backend): A Next.js App Router Route Handler at app/api/chat/route.ts. Receives the messages array from the client, loads the system prompt from Supabase app_config, calls Anthropic SDK streamText() or OpenAI SDK with stream:true, and returns a ReadableStream. Runs server-side so ANTHROPIC_API_KEY never reaches the client bundle.
- **Conversation memory** (data): A Supabase conversations table storing each conversation's messages as a JSONB array. Loaded on component mount and passed to useChat() as initialMessages. After each AI response completes, the updated messages array is upserted back to Supabase. A conversations list sidebar shows the user's previous chats by title (auto-generated from the first user message).
- **System prompt configuration** (data): A Supabase app_config table with a chatbot_system_prompt text column that the API route fetches on every request. This lets you update the chatbot's personality, instructions, and knowledge scope from an admin panel without redeploying. Cache the config in-process for 60 seconds to avoid a DB hit per message.
- **Rate limiting** (backend): Upstash Redis @upstash/ratelimit with a sliding window algorithm: 20 messages per authenticated user per hour. The rate limiter runs at the top of the API route before any LLM call. Returns a 429 response with a 'Rate limit exceeded' message the UI displays to the user. Unauthenticated requests are limited by IP address at a lower threshold (5 per hour).

## Data model

Two tables: conversations (stores message history as JSONB) and chat_usage (tracks token consumption per user per day for quota management). Run this in the Supabase SQL Editor.

```sql
create table public.conversations (
  id uuid primary key default gen_random_uuid(),
  user_id uuid references auth.users(id) on delete cascade not null,
  title text not null default 'New conversation',
  messages jsonb not null default '[]',
  model text not null default 'claude-sonnet-4-5',
  created_at timestamptz not null default now(),
  updated_at timestamptz not null default now()
);

create table public.chat_usage (
  id uuid primary key default gen_random_uuid(),
  user_id uuid references auth.users(id) on delete cascade not null,
  date date not null default current_date,
  message_count integer not null default 0,
  token_count integer not null default 0,
  unique (user_id, date)
);

create table public.app_config (
  key text primary key,
  value text not null,
  updated_at timestamptz not null default now()
);

-- Insert default system prompt
insert into public.app_config (key, value) values
  ('chatbot_system_prompt', 'You are a helpful assistant. Answer questions clearly and concisely. If you are unsure about something, say so.');

-- Indexes
create index conversations_user_idx on public.conversations (user_id, updated_at desc);
create index chat_usage_user_date_idx on public.chat_usage (user_id, date desc);

-- RLS
alter table public.conversations enable row level security;
alter table public.chat_usage enable row level security;
alter table public.app_config enable row level security;

-- Users can only access their own conversations
create policy "Users can view own conversations"
  on public.conversations for select
  using (user_id = auth.uid());

create policy "Users can insert own conversations"
  on public.conversations for insert
  with check (user_id = auth.uid());

create policy "Users can update own conversations"
  on public.conversations for update
  using (user_id = auth.uid());

create policy "Users can delete own conversations"
  on public.conversations for delete
  using (user_id = auth.uid());

-- Chat usage: users see own, admins see all
create policy "Users can view own usage"
  on public.chat_usage for select
  using (user_id = auth.uid());

-- app_config: readable by authenticated users (system prompt is not sensitive)
create policy "Authenticated users can read config"
  on public.app_config for select
  using (auth.uid() is not null);
```

The messages JSONB column stores the full conversation as an array of { role: 'user' | 'assistant', content: string } objects — the exact format Anthropic and OpenAI both expect. The chat_usage table with a unique (user_id, date) constraint lets you increment token counts with an upsert rather than separate insert/update logic.

## Build paths

### V0 — fit 5/10, 3–5 hours

Best path — V0 generates the Vercel AI SDK useChat() hook with streaming out of the box, and Vercel deployment handles API routes as serverless functions without extra configuration.

1. Prompt V0 with the chatbot spec below — it generates the chat UI, the API route with streaming, and the conversation sidebar
2. Add environment variables in V0's Vars panel: ANTHROPIC_API_KEY (server-side, no NEXT_PUBLIC_ prefix), NEXT_PUBLIC_SUPABASE_URL, NEXT_PUBLIC_SUPABASE_ANON_KEY, SUPABASE_SERVICE_ROLE_KEY, UPSTASH_REDIS_REST_URL, and UPSTASH_REDIS_REST_TOKEN
3. Run the SQL schema from this page in the Supabase SQL Editor
4. Create an Upstash Redis database at upstash.com (free tier: 10,000 commands/day) and copy the REST URL and token
5. Deploy to Vercel — streaming works correctly only in the deployed environment, not in V0's preview sandbox

Starter prompt:

```
Build a streaming AI chatbot feature in Next.js App Router using the Vercel AI SDK and Anthropic. API route: app/api/chat/route.ts — import { streamText } from 'ai'; import { anthropic } from '@ai-sdk/anthropic'; export async function POST(req: Request) { const { messages, conversationId } = await req.json(); // Rate limit check using @upstash/ratelimit with Upstash Redis: 20 messages per user per hour, 5 per IP for unauthenticated; return 429 with json { error: 'Rate limit exceeded. Try again in X minutes.' } if exceeded. Load system prompt from Supabase app_config table (key = 'chatbot_system_prompt'). Call streamText({ model: anthropic('claude-sonnet-4-5'), system: systemPrompt, messages, maxTokens: 2048 }); return result.toDataStreamResponse(). After stream completes, upsert conversation to Supabase conversations table (user_id, messages array, updated_at). Chat UI component (client): import { useChat } from 'ai/react'. Chat layout: full-height flex column — top: conversation title + Clear button; middle: ScrollArea with MessageBubble components (user right/blue, AI left/gray), each with role icon and relative timestamp; fade-in animation per message using Framer Motion; auto-scroll to bottom ref on messages change; typing indicator (three animated dots) when isLoading and last message is user role. Bottom: fixed input bar — shadcn/ui Textarea (auto-resize, Enter to submit, Shift+Enter for newline), Send button, Stop button (calls stop() from useChat) shown during isLoading, padding-bottom: env(safe-area-inset-bottom) for iOS keyboard. Empty state: show 4 suggested starter prompts as clickable chips that call append() on click. On mount: fetch conversations list from Supabase (title, updated_at, id) ordered by updated_at DESC; load conversation messages from Supabase by conversationId and pass as initialMessages to useChat. On each AI response complete: auto-generate conversation title from first user message (take first 50 chars). Error handling: show 'I am having trouble responding, please try again' as an AI message bubble on API error. ANTHROPIC_API_KEY must stay server-side — never in a client component.
```

Limitations:

- Streaming does not work in V0's preview sandbox — deploy to Vercel to test the streaming experience
- Supabase setup is manual — V0 does not auto-provision the database; run the SQL schema in the Supabase SQL Editor
- Upstash Redis requires a separate free account — V0 does not provision it

### Lovable — fit 4/10, 4–6 hours

Lovable's AI connector in Cloud tab can wire OpenAI or Anthropic, and Edge Functions handle streaming via Supabase Edge Runtime. Streaming requires an explicit SSE setup which the first AI pass sometimes omits.

1. Create a new Lovable project and connect Lovable Cloud to provision Supabase automatically
2. Open Cloud tab → Secrets and add: ANTHROPIC_API_KEY (your Anthropic API key), UPSTASH_REDIS_REST_URL, and UPSTASH_REDIS_REST_TOKEN
3. Paste the prompt below in Agent Mode — Lovable generates the chat UI and Supabase Edge Function
4. If the streaming response is not working (messages appear all at once after a delay), add a follow-up prompt: 'The Edge Function needs to return a streaming Server-Sent Events response, not a buffered JSON response. Use ReadableStream with a TextEncoder to stream each token as a data: {token} SSE event'
5. Click Publish → Update and test streaming on the live URL — streaming through the Lovable preview iframe is not reliable

Starter prompt:

```
Build a streaming AI chatbot in this Lovable project using Supabase Edge Functions and the Anthropic API. Create a Supabase Edge Function called 'chat' that: 1. Accepts POST { messages: [{role, content}], conversationId: string }. 2. Loads ANTHROPIC_API_KEY from Deno.env.get('ANTHROPIC_API_KEY'). 3. Loads system prompt from Supabase app_config table where key = 'chatbot_system_prompt'. 4. Calls Anthropic API with stream:true, model 'claude-sonnet-4-5', system prompt, and the full messages array. 5. Returns a Server-Sent Events (SSE) ReadableStream — each chunk writes 'data: {token}\n\n', finished with 'data: [DONE]\n\n'. 6. After stream completes, upserts the conversation to the conversations table. Chat UI (React client component): useEffect to call the Edge Function, read the SSE stream, and append tokens to the AI message bubble as they arrive. Show a typing indicator before the first token arrives. ScrollArea auto-scrolls to the latest message. Fixed-bottom Textarea input (Enter to submit, Shift+Enter for newline). Stop button that aborts the fetch via AbortController. Empty conversation state with 4 starter prompt chips. On mount: load conversations list and selected conversation messages from Supabase. Supabase tables needed: conversations (id, user_id, title, messages jsonb, model text, created_at, updated_at), app_config (key, value). RLS: users can only read/write their own conversations. ANTHROPIC_API_KEY must only be accessed in the Edge Function via Deno.env.get() — never in React client code.
```

Limitations:

- Supabase Edge Functions have a 2MB response limit — long conversations with history exceeding this limit need pagination or context window trimming before the Edge Function sends history to Anthropic
- Streaming through Lovable's preview iframe is unreliable — always test on the published URL
- Upstash Redis rate limiting requires a follow-up prompt after the initial build — Lovable does not include it automatically

### Custom — fit 5/10, 3–7 days

Full control over model routing (Haiku for simple queries, Sonnet for complex ones), conversation branching, RAG with pgvector, and per-user quota tiers tied to subscription plans.

1. Build model routing: classify the incoming message complexity with a fast Haiku call first (< 100ms, < $0.01); route simple questions to Haiku, complex multi-step reasoning to Sonnet. Saves 60–70% on LLM costs at scale.
2. Implement conversation branching: each conversation node has a parent_message_id so users can fork from any point in a conversation and explore a different path — like Cursor's conversation branching
3. Add RAG (Retrieval Augmented Generation) with pgvector: embed your product documentation or knowledge base, store vectors in Supabase with pgvector, retrieve top-k relevant chunks per user query, inject into system prompt as context
4. Wire Stripe metered billing: track token_count per user in chat_usage, bill overages above the free tier at the end of each billing period using Stripe Usage Records

Limitations:

- 3–7 day timeline; RAG pipeline alone adds 2–3 days of engineering including embedding generation and vector index tuning
- Model routing with Haiku-as-classifier adds latency on the first classification step — only worthwhile at 1,000+ daily active users where LLM cost optimization matters

## Gotchas

- **Streaming breaks in Lovable preview — messages appear all at once** — Supabase Edge Function SSE streaming does not work reliably through Lovable's preview iframe proxy. The iframe buffers the SSE response and releases it all at once when the stream closes, making the chatbot appear non-streaming during development. This is a preview-only issue — streaming works correctly in production. Fix: Test streaming exclusively on the published URL (Publish → Update in Lovable). Do not judge streaming behavior from the preview pane. If streaming still doesn't work on the live URL, verify the Edge Function is returning a ReadableStream with the correct Content-Type: text/event-stream header.
- **Conversation history exceeds the LLM context window and returns a 400 error** — The useChat() hook sends all stored messages with every request. After 20–30 exchanges, the total token count of the messages array plus the system prompt approaches or exceeds the model's context window. Claude Sonnet has a 200K token context window, but conversations with long messages can hit this surprisingly fast. The API returns a 400 error that the UI usually renders as a generic failure. Fix: Implement a sliding window in the API route: count the tokens of the messages array (roughly 4 characters per token) and trim from the oldest messages until the total fits within 80K tokens (leaving headroom for the system prompt and response). Optionally, summarize the trimmed portion with a quick Haiku call and inject the summary as a system message.
- **V0 generates localStorage conversation persistence that crashes on SSR** — V0 sometimes stores conversation history in localStorage using code at module scope — outside a useEffect. In Next.js App Router, module-scope code runs server-side during the initial render. localStorage is undefined server-side, throwing 'localStorage is not defined' and crashing the page hydration. Fix: Wrap all localStorage access in a useEffect (client-only) or use Supabase for conversation storage (server-safe). The prompt above instructs V0 to load conversation history from Supabase on mount — verify the generated code does not access localStorage before the component mounts.
- **ANTHROPIC_API_KEY exposed in the client bundle** — V0 occasionally generates the Anthropic API call directly inside a React client component using useEffect — the API key is passed as a header from the browser. Any user who opens the browser Network tab can read the key. This leads to immediate key exposure if the app is deployed publicly. Fix: All LLM API calls must go through a Next.js API route (app/api/chat/route.ts) or Supabase Edge Function where the key is only accessible as a server environment variable. In V0: if you see ANTHROPIC_API_KEY referenced in a file with 'use client' at the top, that is the bug. Move the API call to a server route.
- **In-memory rate limiting resets on every serverless cold start** — A common mistake: implementing rate limiting with a Map or object stored in the API route module scope. This works in development (single long-running Node process) but completely fails in production serverless functions — every function invocation potentially gets a fresh cold start, resetting the in-memory counter. Users can exceed rate limits by just waiting for a new cold start. Fix: Always use Upstash Redis for rate limiting in serverless environments. Upstash persists the sliding window counter across all function invocations. The @upstash/ratelimit package handles the sliding window algorithm with a single import and three lines of code.

## Best practices

- Keep ANTHROPIC_API_KEY and OPENAI_API_KEY strictly server-side — never in a client component, never with a NEXT_PUBLIC_ prefix
- Send the full conversation history with every message, but implement a sliding window that trims the oldest messages when the total exceeds 80K tokens
- Load the system prompt from a Supabase config table, not from hardcoded strings — product teams need to adjust chatbot behavior without engineering deploys
- Use Upstash Redis for rate limiting — never in-memory rate limiting in serverless functions
- Implement the Stop button from day one — useChat exposes a stop() function; users who ask a long question and immediately want to rephrase it will expect to be able to cancel
- Generate conversation titles automatically from the first user message (first 50 characters) — users can't remember which of their 20 'New conversation' chats has the information they want
- Handle the 429 rate limit response gracefully in the UI — show the rate limit reset time, not a generic error message

## Frequently asked questions

### How do I add conversation history to my AI chatbot?

Store each conversation's messages as a JSONB array in a Supabase conversations table. On the useChat() hook, pass the loaded messages as initialMessages — the hook sends the full array with every new message, giving the model context. After each AI response completes, upsert the updated messages array back to Supabase. The SQL schema above includes the full conversations table with RLS policies.

### What's the difference between streaming and non-streaming chatbot responses?

Non-streaming waits for the LLM to finish generating the entire response before sending it — users stare at a spinner for 5–30 seconds. Streaming sends each token as it's generated — users see text appearing word by word within 200ms of sending their message. Streaming feels dramatically more responsive. The Vercel AI SDK's useChat() hook handles streaming automatically with ReadableStream.

### How do I prevent users from abusing my chatbot API?

Upstash Redis + @upstash/ratelimit is the standard solution for serverless rate limiting. Configure a sliding window: 20 messages per authenticated user per hour, 5 per IP for unauthenticated users. The rate limit check runs in the API route before any LLM call — a 429 response costs $0 in LLM tokens. Set budget alerts in both Anthropic Console and OpenAI Dashboard as a secondary protection.

### How much does an AI chatbot cost per month?

LLM tokens are the dominant cost. At 10 messages per user per day with 500 average tokens per message: 100 users = approximately $10–15/mo on Claude Sonnet; 1,000 users = $100–150/mo; 10,000 users = $1,000–1,500/mo. Switching to Claude Haiku for simpler queries reduces cost by approximately 85%. Infrastructure (Supabase, Vercel, Upstash) adds $0–50/mo depending on scale.

### Can I use Claude and GPT-4 in the same chatbot?

Yes — the Vercel AI SDK provides a unified interface for both. In the API route, you can call anthropic('claude-sonnet-4-5') for some requests and openai('gpt-4o') for others. A common pattern: use Claude for nuanced, context-rich conversations and GPT-4o-mini for simple factual lookups. Pass the selected model name as a field in the request body and store it in the conversations.model column so each conversation remembers which model was used.

### How do I add a system prompt that defines my chatbot's personality?

Store the system prompt in a Supabase app_config table with a key of 'chatbot_system_prompt'. In the API route, fetch it with `supabase.from('app_config').select('value').eq('key', 'chatbot_system_prompt').single()` and pass it as the system parameter to streamText(). Cache the result for 60 seconds in-process to avoid a DB hit per message. Build an admin UI that lets non-technical team members update the system prompt without redeploying.

### How do I store chat history in Supabase?

After each AI response completes (use the onFinish callback in useChat), call supabase.from('conversations').upsert({ id: conversationId, user_id: userId, messages: updatedMessages, updated_at: new Date().toISOString() }). Load the history on mount: fetch the conversation row by id and pass messages as initialMessages to useChat(). The SQL schema above includes the full conversations table with correct RLS policies.

### How do I stop a chatbot response mid-generation?

The Vercel AI SDK useChat() hook exposes a stop() function that aborts the active ReadableStream. Wire it to a Stop button that appears only when isLoading is true. When stop() is called, the partial AI response so far is preserved in the messages array as the AI's reply — the conversation continues from that point. Do not show the Stop button after the stream completes — toggle it with the isLoading state.

---

Source: https://www.rapidevelopers.com/app-features/ai-chatbot
© RapidDev — https://www.rapidevelopers.com/app-features/ai-chatbot