Feature spec
AdvancedCategory
media-content
Build with AI
2–4 days with Lovable + manual WebXR wiring
Custom build
3–6 weeks custom dev
Running cost
$0–$25/mo up to ~1K users (Supabase free tier)
Works on
Everything it takes to ship Augmented Reality Stickers — parts, prompts, and real costs.
AR stickers need three pieces: a face-landmark engine (MediaPipe FaceMesh — free, in-browser), a canvas overlay anchored to facial landmarks, and a sticker asset CDN. With Lovable you can scaffold the UI shell and Supabase Storage integration in 2–4 days, but WebXR/MediaPipe wiring requires heavy manual JavaScript. Running cost is $0–$25/month for most apps since all computation happens in the browser.
What Augmented Reality Stickers Actually Are
AR stickers overlay animated graphics — hats, glasses, effects — directly onto a live camera feed, anchored to specific points on a user's face. The user sees themselves in real time with the sticker composited on top. Social apps use this for photo booths and filters; marketing campaigns use it for branded selfie experiences. The face-tracking itself (MediaPipe FaceMesh) is open-source and runs entirely in the browser — no cloud API, no per-use cost. The product decisions that make or break the feature are sticker quality, anchor precision, and capture output: a blurry sticker or a drifting overlay at head turn destroys the experience instantly.
What users consider table stakes in 2026
- Real-time sticker tracking anchored to facial landmarks with less than 50 ms perceived lag
- Sticker library browsable by category with thumbnail previews and a search input
- One-tap photo capture that composites the video frame and sticker layer into a downloadable PNG
- Share output via native Web Share API or a generated share link
- Full parity on iOS Safari (front camera, portrait mode) and Android Chrome
- Graceful no-face-found state so the experience doesn't feel broken when the face moves out of frame
Anatomy of the Feature
Six components. The face landmark engine and capture pipeline are where builds fail — AI tools get the React shell right but consistently mis-wire the MediaPipe locateFile path and the canvas compositing order.
Camera access module
UICalls navigator.mediaDevices.getUserMedia() with { video: { facingMode: 'user', width: 1280, height: 720 } } and streams the result into a <video> element. Handles permission denial, device enumeration for front/rear switching, and the iOS Safari portrait rotation bug.
Note: getUserMedia() requires HTTPS and fails silently inside iframes. Always test on a published URL. On iOS Safari, the video stream arrives rotated 90 degrees in portrait — compensate with CSS transform: rotate(-90deg) on the video element and apply the same correction to canvas draw calls.
Face landmark engine
BackendMediaPipe FaceMesh via @mediapipe/face_mesh returns 468 3D facial landmarks at 30 fps. Initialized once per session; drives a requestAnimationFrame loop that feeds video frames to FaceMesh and receives landmark arrays back for each detected face.
Note: The WASM binary (~8 MB) must be loaded from a CDN with an explicit locateFile option. Alternative: face-api.js TinyFaceDetector is smaller but returns only 68 landmarks (not enough for precise sticker anchoring). MediaPipe is the production choice.
Sticker placement engine
UIMaps MediaPipe landmark indices to sticker anchor points and applies affine transforms for rotation and scale. Uses three-point anchoring: landmark 10 (forehead center), landmark 234 (left ear), landmark 454 (right ear) to compute a transform matrix that follows head tilt and turn correctly.
Note: Single-point anchoring (e.g. nose tip only) causes the sticker to drift on head rotation — the most common AI-generated defect. Always request three-point anchoring explicitly in your prompt.
AR rendering layer
UIA <canvas> element positioned absolutely over the <video> feed at identical dimensions. In the requestAnimationFrame loop: clearRect(), draw the transformed sticker image (SVG or WebP) at the computed anchor position using ctx.drawImage() with matrix transforms applied via ctx.setTransform().
Note: All sticker assets must be loaded from the same origin or have CORS headers set — cross-origin images on canvas trigger a SecurityError on toDataURL() calls.
Sticker asset manager
DataSVG, WebP, or APNG sticker sprites stored in a Supabase Storage public bucket; sticker metadata (name, category, tags, is_animated, asset_url) in a Postgres stickers table. Stickers are preloaded as Image() objects at picker open time to avoid per-frame fetch latency.
Note: Serve assets via Supabase's CDN-backed Storage URLs. For animated stickers, APNG degrades gracefully to static on older browsers; Lottie/JSON animations require the lottie-web library which adds 70 KB to the bundle.
Capture and export pipeline
BackendOn capture button press: draw the current video frame onto a temporary OffscreenCanvas, then draw the composited sticker canvas on top, and call toBlob() to produce a PNG Blob. The Blob is either offered as a file download via URL.createObjectURL() or uploaded to Supabase Storage and returned as a signed URL for sharing.
Note: html2canvas cannot capture <video> elements — always use OffscreenCanvas or a secondary canvas that you draw the video frame into manually. For video capture (recording), ffmpeg.wasm can encode the canvas stream to MP4 but adds a 3 MB WASM dependency.
The data model
Two tables: one for the sticker library and one for user captures. Run this in the Supabase SQL editor:
1create table public.stickers (2 id uuid primary key default gen_random_uuid(),3 name text not null,4 category text not null,5 asset_url text not null,6 tags text[] not null default '{}',7 is_animated boolean not null default false,8 created_at timestamptz not null default now()9);1011alter table public.stickers enable row level security;1213create policy "Stickers are publicly readable"14 on public.stickers for select15 using (true);1617create index stickers_category_idx18 on public.stickers (category);1920create table public.user_captures (21 id uuid primary key default gen_random_uuid(),22 user_id uuid references auth.users(id) on delete cascade not null,23 capture_url text not null,24 sticker_ids uuid[] not null default '{}',25 created_at timestamptz not null default now()26);2728alter table public.user_captures enable row level security;2930create policy "Users can view own captures"31 on public.user_captures for select32 using (auth.uid() = user_id);3334create policy "Users can insert own captures"35 on public.user_captures for insert36 with check (auth.uid() = user_id);3738create policy "Users can delete own captures"39 on public.user_captures for delete40 using (auth.uid() = user_id);4142create index user_captures_user_recent_idx43 on public.user_captures (user_id, created_at desc);Heads up: The stickers table uses public read so the picker loads without auth. user_captures restricts all access to the owning user. The category index keeps the sticker picker fast as the library grows.
Build it — pick your path
Each tool fits this feature differently. Switch tabs to compare fit, time, the exact steps, and a copy-paste prompt.
Full control over MediaPipe FaceMesh configuration, WebCodecs for video export, GPU-accelerated canvas rendering, and CDN sticker delivery. Custom dev is the only reliable path to production-quality AR at scale with sub-30 ms latency and multi-face support.
Step by step
- 1Set up Next.js with @mediapipe/face_mesh, configure locateFile for WASM CDN delivery, and build the requestAnimationFrame inference loop with proper cleanup on unmount
- 2Implement three-point affine transform anchoring using landmarks 10, 234, and 454 for accurate sticker placement across head orientations
- 3Build the sticker asset pipeline: WebP sprites in Supabase Storage, metadata in Postgres, preloaded Image() objects keyed by sticker ID
- 4Add WebCodecs-based video recording that captures the composited canvas stream as MP4 for video sticker output
- 5Implement CDN caching strategy for sticker assets and signed URL generation for user captures
Where this path bites
- 3–6 week timeline requires engineers with WebGL/WebXR expertise
- MediaPipe WASM initialization adds 1–2 seconds on first load — requires a loading screen
Third-party services you'll need
Face tracking and rendering are free — the only recurring cost is asset and capture storage:
| Service | What it does | Free tier | Paid from |
|---|---|---|---|
| MediaPipe FaceMesh | In-browser face landmark detection at 30 fps — the core tracking engine | Fully free, open-source, runs in-browser (no API cost) | Free |
| Supabase Storage | Hosts sticker asset files and user capture PNGs | 1 GB storage included in free tier | Pro $25/mo includes 100 GB (approx) |
| Cloudflare R2 | Alternative CDN for sticker assets at higher scale | 10 GB/mo free | $0.015/GB thereafter (approx) |
Swipe the table sideways to see pricing.
What it costs to run
Drag through the tiers to see how your monthly bill scales with users — no surprises later.
Estimated monthly running cost
MediaPipe runs in-browser (no API cost). Supabase free tier (1 GB) covers sticker assets and captures comfortably. Bandwidth stays under free limits.
Estimates use verified 2026 pricing and assume typical usage per user. Your real bill depends on activity, storage, and third-party plans.
What breaks when AI tools build this
The failures people actually hit on their first build — the symptom, why it happens, and the exact fix.
Camera blocked in preview iframe
Symptom: getUserMedia() silently fails or throws NotAllowedError inside Lovable's embedded preview and V0's sandbox iframe. Browser security policies do not grant microphone or camera permissions to sandboxed iframes. The video element remains black and MediaPipe never initializes, making it impossible to validate the feature in the builder's preview panel.
Fix: Always test AR sticker features on the published HTTPS URL opened on a real device. In Lovable, use the mobile preview QR code; in V0, publish to production first. Add a clear UI indicator ('Open on your phone to test AR') in the preview state.
MediaPipe WASM fails to load
Symptom: @mediapipe/face_mesh requires its WASM binary (~8 MB) to be fetched at runtime. ESM bundlers (Vite, esbuild, webpack) often fail to resolve the locateFile path correctly, resulting in a 'Failed to fetch' error that prevents FaceMesh from initializing. The error appears in the console but the UI may show a blank camera feed with no indication of what went wrong.
Fix: Explicitly set the locateFile option when constructing FaceMesh: locateFile: (file) => `https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/${file}`. This bypasses the bundler and fetches WASM directly from the jsDelivr CDN. Test by opening DevTools Network tab and confirming the .wasm file loads with a 200 response.
Sticker drifts when head rotates
Symptom: AI-generated AR code commonly anchors the sticker to a single landmark (e.g. landmark 1 — nose tip). When the user turns their head, the sticker's reference point moves but its orientation doesn't update, causing the graphic to visibly drift away from the intended position. At large head angles the sticker can float completely off the face.
Fix: Use three-point anchoring with landmarks 10 (forehead center), 234 (left ear), and 454 (right ear) to compute an affine transform matrix. Apply the matrix via ctx.setTransform() before drawing the sticker. This approach correctly accounts for head rotation and scale changes from distance. Specify this explicitly in your prompt — single-point anchoring is the default AI output.
iOS Safari captures appear rotated
Symptom: iOS Safari delivers the front camera video stream in landscape orientation regardless of the physical device orientation. When you draw that video frame to a canvas in portrait mode and call toDataURL(), the resulting PNG appears rotated 90 degrees. This produces unusable captures and only manifests on iPhone — desktop and Android are unaffected.
Fix: Detect iOS with a user agent check and apply CSS transform: rotate(-90deg) to the video element. In the capture canvas drawImage() call, rotate the canvas context 90 degrees before drawing the video frame, then rotate back. Test captures on a real iPhone before launch — the Lovable or V0 desktop preview will look correct even when the mobile output is broken.
Canvas export throws SecurityError on sticker assets
Symptom: Drawing sticker images loaded from a different origin onto a canvas and calling toDataURL() or toBlob() throws a SecurityError ('Tainted canvases may not be exported'). This happens when sticker assets are served from Supabase Storage but the canvas origin is your app domain. The capture button appears to work until the export step, then silently fails.
Fix: Serve all sticker assets from your app's same origin, or configure CORS headers on the Supabase Storage bucket. In the Supabase Dashboard, go to Storage → Policies and add a CORS configuration allowing your app's origin. Also set crossOrigin='anonymous' on the Image() objects used to preload stickers.
Best practices
Always use three-point landmark anchoring (forehead 10, left ear 234, right ear 454) — single-point anchoring drifts and makes the feature feel broken on head turn
Preload all sticker Image() objects when the picker opens, not on selection — per-frame fetch latency makes the overlay visibly lag during the first second
Show a distinct 'no face detected' state with an animated icon rather than a static camera feed — users don't know if the feature is broken or just waiting
Cancel the requestAnimationFrame loop and stop the media stream track when the component unmounts to prevent memory leaks and lingering camera indicator lights
Serve sticker assets from Supabase Storage with a public bucket and CORS headers set from day one — fixing CORS after captures are broken is painful
Add an explicit autoplay attribute to the <video> element (autoplay muted playsInline) — iOS Safari requires playsInline to avoid full-screen hijack
Compress sticker assets aggressively: WebP stickers at 150x150 px should be under 20 KB each; animated APNG under 100 KB — a slow sticker picker kills the experience
Log the MediaPipe initialization time in development — if it exceeds 3 seconds, the locateFile CDN path is wrong and WASM is being bundled instead of fetched
When You Need Custom Development
AI tools can scaffold the UI shell and Supabase integration, but AR sticker quality at production scale requires custom engineering:
- You need sub-30 ms AR latency for live video streaming, not just photo capture — this requires WebCodecs and GPU-accelerated canvas rendering beyond what AI scaffolds produce
- The experience requires multi-face tracking for group AR scenes (friend photos, party mode) — MediaPipe supports multiple faces but the sticker placement logic must be hand-written per face
- Stickers are 3D meshes (GLB/GLTF) rather than flat sprites — this requires Three.js or Babylon.js integration with face mesh rigging
- You need to integrate a proprietary face-tracking SDK such as Snap Lens Studio or 8th Wall, which have their own WebXR pipelines incompatible with MediaPipe
RapidDev builds these features for production
Real auth, database, and payments — fixed price, delivered in weeks. You own 100% of the code.
Frequently asked questions
Can I add AR stickers without WebXR?
Yes — and for face-tracked stickers on a flat camera feed, you don't need WebXR at all. WebXR is for immersive mixed-reality experiences (headsets, AR overlays on real-world geometry). MediaPipe FaceMesh running on a standard getUserMedia() video stream is the correct approach for social-style AR sticker filters. WebXR Device API adds complexity and has worse mobile browser support.
Do AR stickers work on iPhone Safari?
Yes, but with caveats. iOS Safari supports getUserMedia() for camera access and runs MediaPipe FaceMesh at usable frame rates on iPhone 12 and newer. Two gotchas: the video stream arrives rotated in portrait mode (requires a CSS + canvas transform fix), and Safari caps resolution at 720p. The feature works well for photo capture; video recording requires careful codec selection because MediaRecorder support in Safari is limited.
How do I anchor a sticker to the forehead vs the nose?
MediaPipe FaceMesh returns 468 numbered landmarks. Landmark 10 is the forehead center, landmark 1 is the nose tip, landmark 33 is the left eye outer corner, and landmark 263 is the right eye outer corner. To place a sticker at a specific location, use the (x, y, z) values of the target landmark as the draw position and derive rotation from the vector between landmarks 33 and 263 (eye line angle). For a hat, landmark 10 with an upward offset proportional to face height gives the most natural placement.
What's the difference between face-api.js and MediaPipe FaceMesh?
face-api.js TinyFaceDetector returns 68 landmarks and is lighter (~1.5 MB), but 68 points aren't enough for precise sticker placement — hat brims and glasses frames warp noticeably. MediaPipe FaceMesh returns 468 3D landmarks and is the industry standard for face AR in browsers, at the cost of a larger WASM payload (~8 MB initial load). For sticker overlays, use MediaPipe FaceMesh.
How do I let users upload their own stickers?
Add an upload button that sends the file to a Supabase Storage bucket (e.g. 'user-stickers/{user_id}/') via the Supabase JS client, then inserts a row in the stickers table with the returned URL. Restrict uploads to SVG, WebP, and PNG under 500 KB. Apply the same CORS configuration to the user-stickers bucket as to the system stickers bucket, or captures will throw SecurityError on export.
Can users record a video with AR stickers, not just a photo?
Yes, but it adds significant complexity. Use MediaRecorder with the canvas.captureStream(30) API to record the composited canvas as a video stream. Export as WebM (widely supported) and optionally transcode to MP4 using ffmpeg.wasm. The ffmpeg.wasm dependency adds about 3 MB to your bundle. Safari has limited MediaRecorder support, so test on both Chrome and Safari before promising video capture.
How do I handle the 'no face detected' state gracefully?
Track a consecutive-frames-without-face counter in the requestAnimationFrame loop. After 30 frames (~1 second) with no landmarks returned, set state to 'no-face-found' and show an animated prompt (e.g. 'Move your face into frame'). Reset the counter and hide the prompt the instant FaceMesh returns landmarks again. Avoid showing the no-face state immediately — a brief look away shouldn't disrupt the experience.
What frame rate should I target for smooth AR sticker rendering?
Target 30 fps, which MediaPipe FaceMesh can sustain on mid-range phones from 2021 onward. The requestAnimationFrame loop runs at the display refresh rate (60 Hz typical), so process FaceMesh inference every other frame on slower devices. Add a performance monitor during testing: if inference time exceeds 20 ms per frame, enable the MediaPipe FaceMesh modelComplexity: 0 option (simpler model, slightly fewer landmarks) to reduce CPU load.
Need this feature production-ready?
RapidDev builds augmented reality stickers into real apps — auth, database, payments — at $13K–$25K.
Book a free consultation30-min call. No commitment.