# How to Build a Recommendations Engine with Replit

- Tool: How to Build with Replit
- Difficulty: Advanced
- Compatibility: Replit Free
- Last updated: April 2026

## TL;DR

Build a collaborative filtering recommendations engine in Replit using Express and PostgreSQL in 2-4 hours. You'll track user interactions, compute item similarity scores with a nightly Scheduled Deployment, and serve personalized recommendations via a fast cache layer — a lightweight Netflix-style system without any ML infrastructure.

## Before you start

- A Replit account (Free tier is sufficient for under 10K items)
- An existing app or catalog you want to add recommendations to (products, articles, videos, etc.)
- Basic understanding of what collaborative filtering means (users who liked X also liked Y)
- No external API keys needed

## Step-by-step guide

### 1. Scaffold the project with Replit Agent

Create a new Replit App and paste this prompt. Agent builds the complete recommendations backend with all five tables and core API routes.

```
// Build a recommendations engine with Express and PostgreSQL using Drizzle ORM.
//
// Tables:
// 1. items: id serial primary key, name text not null, category text,
//    tags text[] (PostgreSQL array), metadata jsonb, created_at timestamp default now()
// 2. users: id serial primary key, user_id text not null unique,
//    preferences jsonb, created_at timestamp default now()
// 3. interactions: id serial primary key, user_id integer references users not null,
//    item_id integer references items not null, type text not null
//    (enum: view/click/purchase/rating/wishlist), value numeric (1-5 for ratings),
//    weight numeric default 1.0, created_at timestamp default now()
// 4. similarity_scores: id serial primary key, item_a_id integer references items not null,
//    item_b_id integer references items not null, score numeric not null (0 to 1),
//    algorithm text default 'cosine', calculated_at timestamp default now(),
//    unique(item_a_id, item_b_id, algorithm)
// 5. recommendation_cache: id serial primary key, user_id integer references users not null unique,
//    recommendations jsonb not null (array of {item_id, score, reason}),
//    calculated_at timestamp default now()
//
// Routes:
// POST /api/interactions (record interaction — no auth required, just user_id in body)
// GET /api/recommendations/:userId (get personalized recommendations, uses cache)
// GET /api/items/:id/similar (get similar items from similarity_scores)
// GET /api/popular (globally popular items fallback)
// POST /api/admin/recalculate (trigger similarity matrix rebuild — requires ADMIN_KEY header)
//
// Use Replit Auth for admin routes. Bind to 0.0.0.0:3000.
```

> Pro tip: Seed the items table with at least 20 test items before testing recommendations. Without a catalog, there's nothing to recommend. Ask Agent: 'Insert 20 sample items into the items table with varied categories and tags.'

**Expected result:** Running Express app with all five tables. GET /api/popular returns an empty array until interactions are recorded.

### 2. Build the interaction recording API

The interactions endpoint is called every time a user does something with an item. Different interaction types have different weights — a purchase signals much stronger preference than a view.

```
const express = require('express');
const { db } = require('../db');
const { users, interactions } = require('../schema');
const { eq } = require('drizzle-orm');
const { withDbRetry } = require('../lib/retryDb');

const router = express.Router();

// Interaction weights — higher = stronger signal
const WEIGHTS = {
  view:     0.5,
  click:    1.0,
  wishlist: 1.5,
  rating:   2.0,
  purchase: 3.0,
};

router.post('/api/interactions', express.json(), async (req, res) => {
  const { userId, itemId, type, value } = req.body;

  if (!userId || !itemId || !type) {
    return res.status(400).json({ error: 'userId, itemId, and type are required' });
  }
  if (!WEIGHTS[type]) {
    return res.status(400).json({ error: `Invalid type. Must be: ${Object.keys(WEIGHTS).join(', ')}` });
  }

  // Upsert the user record (create if first interaction)
  await withDbRetry(() =>
    db.insert(users)
      .values({ userId: String(userId) })
      .onConflictDoNothing()
  );

  // Get internal user ID
  const [user] = await db.select().from(users)
    .where(eq(users.userId, String(userId))).limit(1);

  const weight = WEIGHTS[type];

  await withDbRetry(() =>
    db.insert(interactions).values({
      userId: user.id,
      itemId: parseInt(itemId),
      type,
      value: value ? parseFloat(value) : null,
      weight,
    })
  );

  return res.json({ recorded: true });
});

module.exports = router;
```

> Pro tip: Call POST /api/interactions from your main app whenever a user views, clicks, or purchases an item. Send it as a fire-and-forget request so tracking never slows down the user experience. In the browser: fetch('/api/interactions', {method:'POST', ...}).catch(() => {})

**Expected result:** POST /api/interactions with {userId: 'u1', itemId: 5, type: 'view'} returns {recorded: true} and inserts into the interactions table with weight 0.5.

### 3. Build the similarity calculation batch job

The similarity matrix is pre-computed nightly by a Scheduled Deployment. This PostgreSQL function calculates cosine similarity between all item pairs based on their interaction vectors.

```
// server/jobs/calculateSimilarity.js — run as a Scheduled Deployment nightly
const { db } = require('../db');
const { similarityScores } = require('../schema');
const { sql } = require('drizzle-orm');

async function calculateSimilarityMatrix() {
  console.log('[similarity] Starting similarity matrix calculation...');

  // Calculate cosine similarity between all item pairs
  // that share at least 3 common users (reduces noise)
  await db.execute(sql`
    INSERT INTO similarity_scores (item_a_id, item_b_id, score, algorithm, calculated_at)
    SELECT
      a.item_id AS item_a_id,
      b.item_id AS item_b_id,
      -- Cosine similarity: SUM(wa*wb) / (SQRT(SUM(wa^2)) * SQRT(SUM(wb^2)))
      SUM(a.weight * b.weight) /
        (SQRT(SUM(a.weight * a.weight)) * SQRT(SUM(b.weight * b.weight))) AS score,
      'cosine' AS algorithm,
      NOW() AS calculated_at
    FROM interactions a
    JOIN interactions b
      ON a.user_id = b.user_id
      AND a.item_id < b.item_id  -- only compute each pair once
    GROUP BY a.item_id, b.item_id
    HAVING COUNT(DISTINCT a.user_id) >= 2  -- at least 2 shared users
      AND SUM(a.weight * a.weight) > 0
      AND SUM(b.weight * b.weight) > 0
    ON CONFLICT (item_a_id, item_b_id, algorithm)
    DO UPDATE SET
      score = EXCLUDED.score,
      calculated_at = EXCLUDED.calculated_at
  `);

  // Update recommendation cache for all active users
  const usersResult = await db.execute(sql`SELECT id, user_id FROM users LIMIT 1000`);
  for (const user of usersResult.rows) {
    await buildUserRecommendations(user.id, user.user_id);
  }

  console.log('[similarity] Done.');
}

async function buildUserRecommendations(internalUserId, externalUserId) {
  // Get items the user has already interacted with
  const userItems = await db.execute({
    sql: 'SELECT DISTINCT item_id FROM interactions WHERE user_id = $1',
    params: [internalUserId],
  });
  const seen = new Set(userItems.rows.map(r => r.item_id));

  if (seen.size === 0) return; // Skip cold-start users

  // Find similar items the user hasn't seen yet
  const recs = await db.execute({
    sql: `
      SELECT
        CASE WHEN ss.item_a_id = ANY($1::int[]) THEN ss.item_b_id ELSE ss.item_a_id END AS rec_item_id,
        MAX(ss.score) AS score,
        MIN(CASE WHEN ss.item_a_id = ANY($1::int[]) THEN ss.item_a_id ELSE ss.item_b_id END) AS because_item_id
      FROM similarity_scores ss
      WHERE (ss.item_a_id = ANY($1::int[]) OR ss.item_b_id = ANY($1::int[]))
        AND NOT (ss.item_a_id = ANY($1::int[]) AND ss.item_b_id = ANY($1::int[]))
      GROUP BY rec_item_id
      ORDER BY score DESC
      LIMIT 20
    `,
    params: [Array.from(seen)],
  });

  const recommendations = recs.rows.map(r => ({
    item_id: r.rec_item_id,
    score: parseFloat(r.score),
    because_item_id: r.because_item_id,
  }));

  await db.execute({
    sql: `
      INSERT INTO recommendation_cache (user_id, recommendations, calculated_at)
      VALUES ($1, $2, NOW())
      ON CONFLICT (user_id) DO UPDATE SET
        recommendations = EXCLUDED.recommendations,
        calculated_at = EXCLUDED.calculated_at
    `,
    params: [internalUserId, JSON.stringify(recommendations)],
  });
}

calculateSimilarityMatrix()
  .then(() => process.exit(0))
  .catch(err => { console.error('[similarity] Error:', err); process.exit(1); });
```

> Pro tip: In Replit's Publish pane, create a Scheduled Deployment for this job set to run at 2:00 AM daily. Set the entrypoint to server/jobs/calculateSimilarity.js. The first run will be slow if you have many items — subsequent runs are faster because of the ON CONFLICT DO UPDATE upsert.

**Expected result:** After the job runs, the similarity_scores table fills with cosine similarity values between item pairs. recommendation_cache shows pre-ranked recommendations per user.

### 4. Build the recommendations retrieval API

The recommendation endpoint serves from the pre-built cache for fast response. For cold-start users with no interactions, it falls back to the globally popular items.

```
const express = require('express');
const { db } = require('../db');
const { users, recommendationCache, items, interactions } = require('../schema');
const { eq, sql, desc } = require('drizzle-orm');

const router = express.Router();

router.get('/api/recommendations/:userId', async (req, res) => {
  const { userId } = req.params;

  // Find internal user record
  const [user] = await db.select().from(users)
    .where(eq(users.userId, userId)).limit(1);

  if (!user) {
    // New user — return popular items (cold start)
    return servePopular(res);
  }

  // Check cache freshness (accept if calculated in last 24 hours)
  const [cache] = await db.select().from(recommendationCache)
    .where(eq(recommendationCache.userId, user.id)).limit(1);

  if (cache && new Date() - new Date(cache.calculatedAt) < 24 * 60 * 60 * 1000) {
    return res.json({
      userId,
      source: 'cache',
      recommendations: cache.recommendations,
    });
  }

  // Cache stale or missing — return popular as fallback until next nightly run
  return servePopular(res);
});

async function servePopular(res) {
  const popular = await db.execute(sql`
    SELECT item_id, COUNT(*) AS interaction_count, COUNT(DISTINCT user_id) AS user_count
    FROM interactions
    GROUP BY item_id
    ORDER BY user_count DESC, interaction_count DESC
    LIMIT 10
  `);

  return res.json({
    source: 'popular',
    recommendations: popular.rows.map(r => ({
      item_id: r.item_id,
      score: null,
      reason: 'Popular with other users',
    })),
  });
}

router.get('/api/items/:id/similar', async (req, res) => {
  const itemId = parseInt(req.params.id);
  const similar = await db.execute({
    sql: `
      SELECT
        CASE WHEN item_a_id = $1 THEN item_b_id ELSE item_a_id END AS similar_item_id,
        score
      FROM similarity_scores
      WHERE item_a_id = $1 OR item_b_id = $1
      ORDER BY score DESC
      LIMIT 8
    `,
    params: [itemId],
  });
  return res.json({ itemId, similar: similar.rows });
});

module.exports = router;
```

> Pro tip: Enrich the recommendation response by joining item IDs against the items table before returning. The cache stores just IDs and scores for compactness — expand them in the route handler to return full item objects with name, image_url, and category.

**Expected result:** GET /api/recommendations/u1 returns personalized recommendations from cache. A new user gets popular items instead. GET /api/items/5/similar returns the 8 most similar items.

### 5. Build the React recommendations widget

Ask Agent to build the frontend recommendations widget and admin dashboard that shows the similarity matrix heatmap.

```
// Ask Agent to build the React frontend with this prompt:
// Build a React recommendations UI with two components:
//
// 1. RecommendationsWidget component:
//    - Accept userId prop
//    - Fetch GET /api/recommendations/:userId on mount
//    - Display a horizontal scrollable row of item cards
//    - Each card shows: item image (from metadata.image_url), item name, category badge,
//      match score as percentage (score * 100, rounded)
//    - If recommendations have because_item_id, show a tooltip on hover:
//      'Because you interacted with [item name]'
//    - Show 'Popular picks' heading if source='popular' (cold start state)
//
// 2. Admin Dashboard page at /admin/recommendations:
//    - Require ADMIN_KEY header (store in localStorage)
//    - Fetch GET /api/admin/recalculate stats: count of items, interactions, similarity pairs
//    - Show three stat cards: Total Items, Total Interactions, Similarity Pairs Computed
//    - Show a 'Recalculate Now' button that calls POST /api/admin/recalculate
//    - Show a heatmap-style similarity matrix: sample 20 random items,
//      show a 20x20 grid where each cell color = similarity score (white=0, blue=1)
//    - Show recent calculation timestamp
//
// Also add a GET /api/items route that returns items with their recommendation stats
// (interaction_count, unique_users from joining interactions).
```

**Expected result:** The RecommendationsWidget renders a horizontal card row with match scores. The admin dashboard shows system stats and a colored similarity matrix grid.

## Complete code example

File: `server/routes/interactions.js`

```javascript
const express = require('express');
const { db } = require('../db');
const { users, interactions, items } = require('../schema');
const { eq } = require('drizzle-orm');
const { withDbRetry } = require('../lib/retryDb');

const router = express.Router();

const INTERACTION_WEIGHTS = {
  view: 0.5,
  click: 1.0,
  wishlist: 1.5,
  rating: 2.0,
  purchase: 3.0,
};

router.post('/api/interactions', express.json(), async (req, res) => {
  const { userId, itemId, type, value } = req.body;
  if (!userId || !itemId || !type) {
    return res.status(400).json({ error: 'userId, itemId, and type are required' });
  }
  if (!INTERACTION_WEIGHTS[type]) {
    return res.status(400).json({
      error: `Invalid type. Valid: ${Object.keys(INTERACTION_WEIGHTS).join(', ')}`,
    });
  }
  // Verify item exists
  const [item] = await db.select({ id: items.id })
    .from(items).where(eq(items.id, parseInt(itemId))).limit(1);
  if (!item) return res.status(404).json({ error: 'Item not found' });

  // Upsert user
  await withDbRetry(() =>
    db.insert(users).values({ userId: String(userId) }).onConflictDoNothing()
  );
  const [user] = await db.select().from(users)
    .where(eq(users.userId, String(userId))).limit(1);

  await withDbRetry(() =>
    db.insert(interactions).values({
      userId: user.id,
      itemId: item.id,
      type,
      value: value != null ? parseFloat(value) : null,
      weight: INTERACTION_WEIGHTS[type],
    })
  );

  return res.json({ recorded: true });
});

router.get('/api/popular', async (req, res) => {
  const result = await db.execute({
    sql: `
      SELECT i.id, i.name, i.category, i.tags, i.metadata,
        COUNT(int.id) AS interaction_count,
        COUNT(DISTINCT int.user_id) AS user_count
      FROM items i
      LEFT JOIN interactions int ON i.id = int.item_id
      GROUP BY i.id
module.exports = router;
```

## Common mistakes

- **Calculating cosine similarity in real time on every recommendation request** — With 10K items, computing all pairwise similarities on demand takes minutes. The API would time out on every request. Fix: Always pre-compute and store similarity_scores via the nightly Scheduled Deployment. Recommendation API reads from similarity_scores and recommendation_cache — both are indexed lookups.
- **Not handling the cold-start problem for new users** — A new user with zero interactions would get an empty recommendations response, which is a broken experience. Fix: Always check if the user has interactions before querying the cache. Return popular items (GET /api/popular) as the fallback for users with fewer than 3 interactions.
- **Recommending items the user already interacted with** — Recommending products someone already bought frustrates users and wastes recommendation slots. Fix: The recommendation builder queries exclude items already in the user's interaction history using WHERE NOT (item_id = ANY(seen_item_ids)).
- **Running the similarity job without a PostgreSQL retry wrapper** — The nightly batch job runs at 2 AM when the database may have been idle for hours and is in sleep mode. The first query will fail, and without retry logic, the entire job fails silently. Fix: Wrap the first DB call in withDbRetry() to handle the cold-start connection failure. The retry with 250ms delay is enough for the database to wake up.

## Best practices

- Pre-compute similarity_scores nightly via Scheduled Deployment — never calculate cosine similarity on the request path.
- Use the recommendation_cache table for instant recommendation serving — check if cache is less than 24 hours old before falling back to real-time computation.
- Weight different interaction types differently: purchases (3.0) >> ratings (2.0) >> wishlist (1.5) >> click (1.0) >> view (0.5). Purchase signals are 6x more valuable than views.
- Keep the item catalog under 10K items with Replit's free PostgreSQL. The O(N^2) similarity calculation becomes slow above this — move to a dedicated vector database for larger catalogs.
- Use Drizzle Studio (Database tab) to inspect similarity_scores after the first batch job run — verify that similar items are being grouped logically before wiring up the frontend.
- Add a minimum shared-users threshold (at least 2) to the similarity calculation to filter out coincidental single-user overlap that would produce artificially high similarity scores.
- Deploy on Autoscale for the main API (recommendations serve from cache = fast). The nightly batch job runs as a separate Scheduled Deployment on Reserved VM for reliability.

## Frequently asked questions

### How many items can this handle before it gets too slow?

With Replit's free PostgreSQL and the pre-computed similarity approach, 10K items works well. The nightly similarity job computes up to 50M pairs (10K^2 / 2) — this takes 10-30 minutes but runs offline. The recommendation API always reads from the cache so users never see slowdowns.

### What's the cold-start problem and how is it handled?

New users have no interaction history, so there's nothing to base personalized recommendations on. The engine handles this by returning globally popular items (sorted by unique_users from the interactions table) for users with zero or very few interactions. Once they have 3+ interactions, the nightly job will build their personalized cache.

### Can I use this to recommend content (articles, videos) instead of products?

Yes — the engine is content-agnostic. The items table has a metadata JSONB column for any attributes. Track 'view' interactions when users read articles, 'rating' when they rate videos. The collaborative filtering algorithm works identically regardless of what type of item is being recommended.

### What Replit plan do I need?

Free tier for the main API (Autoscale). The nightly Scheduled Deployment also runs on Free tier. For large catalogs (5K+ items), the similarity calculation benefits from Reserved VM to avoid the 30-second cold start of the job triggering at 2 AM.

### How often should I run the similarity recalculation?

Nightly is a good default for most apps. If your catalog changes frequently (new items added hourly) or your users are highly active, run it every 6 hours. For very small or slowly-changing catalogs, weekly is sufficient.

### Should I deploy on Autoscale or Reserved VM?

Autoscale for the main recommendation API (cache reads are fast and handle cold starts). Reserved VM for the nightly batch job — you want it to start reliably at 2 AM without cold-start delays causing timeouts on the similarity computation.

### Can RapidDev help build a recommendations engine for my platform?

Yes. RapidDev has built recommendation systems for e-commerce and content platforms across 600+ client projects. We can extend this engine with content-based filtering, A/B testing of recommendation strategies, and real-time similarity updates. Free consultation available.

### How is this different from just sorting by 'most popular'?

Popularity sorting shows the same items to everyone — the top sellers dominate. Collaborative filtering personalizes results based on what similar users liked. A user who buys niche products gets recommendations tailored to their taste, not just bestsellers. Personalization increases click-through rates by 2-5x over simple popularity sorting in typical e-commerce apps.

---

Source: https://www.rapidevelopers.com/how-to-build-replit/recommendations-engine
© RapidDev — https://www.rapidevelopers.com/how-to-build-replit/recommendations-engine