# Apache Kafka

- Tool: FlutterFlow
- Difficulty: Advanced
- Time required: 90 minutes
- Last updated: July 2026

## TL;DR

Connect FlutterFlow to Apache Kafka by pointing a FlutterFlow API Group at a Confluent REST Proxy or a Firebase Cloud Function producer wrapper — FlutterFlow cannot speak Kafka's native TCP protocol directly. For consuming events, run a backend consumer service that writes Kafka events into Firestore or Supabase Realtime, then subscribe your FlutterFlow UI to that downstream store for live updates.

## Kafka in FlutterFlow: REST Proxy + Realtime Fan-Out

Apache Kafka is an event-streaming engine designed around a binary TCP protocol — clients connect to brokers on port 9092, join consumer groups, and commit offsets. A Flutter app compiled by FlutterFlow runs on users' phones and in the browser; it cannot open raw TCP sockets to Kafka brokers, and even if it could, you would never want Kafka bootstrap credentials (SASL username/password, Confluent API key/secret) embedded in a downloadable binary.

The architectural answer has two halves. For producing messages (the app sending events to Kafka), you place an HTTP bridge in front of the broker: Confluent Cloud's built-in REST Proxy exposes `POST /topics/{topic}` over HTTPS, or you can write a Firebase Cloud Function that wraps a Kafka producer and accepts a simple JSON payload from FlutterFlow. FlutterFlow's API Group calls that HTTPS endpoint — the Confluent API key/secret stays in the proxy or Cloud Function config, never in the app.

For consuming messages (the app receiving events from Kafka), the picture is different. The Confluent REST Proxy does have consumer-group endpoints, but they are stateful — each phone would need to maintain its own consumer instance, manage offsets, and poll regularly. This pattern leaks memory server-side and is fragile on mobile where network connections drop. The recommended pattern instead is a persistent backend consumer (a Cloud Function on a schedule, a Cloud Run service, or a simple Node.js process) that subscribes to Kafka topics and writes each event into Firestore or Supabase Realtime. FlutterFlow then subscribes to that downstream store natively, getting real-time updates without knowing Kafka exists. Confluent Cloud's free tier includes roughly $400 in credit to get started — check current pricing at confluent.io for exact details.

## Before you start

- A Confluent Cloud account (or a self-hosted Kafka cluster with REST Proxy enabled) OR a Firebase project to host a Kafka producer Cloud Function
- A Kafka topic already created in your cluster
- Confluent API key and secret (for Confluent Cloud) stored in your backend — never in FlutterFlow
- A Firebase or Supabase project for the downstream Realtime store (for the consumer path)
- A FlutterFlow project with Firebase or Supabase already integrated (for native real-time subscriptions)

## Step-by-step guide

### 1. Understand the architecture before you build

Before touching FlutterFlow, it is worth being explicit about what you are building and why. Kafka is not a request-response database — it is an append-only log. You produce events into topics and consume them in order. This event-log model is powerful for data pipelines but does not map naturally onto a mobile app's interaction model, where the user taps a button and expects an immediate response.

The architecture for FlutterFlow has two distinct data flows: outbound (FlutterFlow → Kafka) and inbound (Kafka → FlutterFlow). For outbound, you need an HTTP endpoint that accepts a JSON payload from FlutterFlow and forwards it to Kafka as a produced message — this is the REST Proxy or Cloud Function producer wrapper. For inbound, you need a persistent backend process that consumes Kafka events and writes them to a real-time store (Firestore or Supabase) that FlutterFlow can subscribe to.

If you only need the outbound direction (e.g. logging user actions to Kafka), the setup is simpler — just the Cloud Function producer. If you need both directions (e.g. showing live order updates), you need the full architecture with a backend consumer. Plan which use-case you are building before continuing.

**Expected result:** You have a clear picture of which components you need to build based on whether you need produce-only, consume-only, or both directions.

### 2. Set up a Kafka producer endpoint (Cloud Function or REST Proxy)

The cleanest approach for most FlutterFlow apps is a Firebase Cloud Function that wraps a Kafka producer using the `kafkajs` npm package. This gives you full control over error handling, authentication, and payload validation without exposing any Kafka credentials to the client.

Create a new Cloud Function. Install `kafkajs` as a dependency. Initialise the Kafka client with your Confluent Cloud bootstrap URL and SASL credentials — stored in Firebase Functions config, not hardcoded. The function accepts a POST request with a JSON body containing `topic`, `key` (optional), and `value` (the event payload), creates a producer, sends the message, and returns a success response.

Alternatively, if you are using Confluent Cloud, you can call the built-in REST Proxy directly from FlutterFlow's API Group. The REST Proxy URL for Confluent Cloud looks like `https://pkc-xxxxx.region.provider.confluent.cloud:443`. You'll need to set the `Content-Type` header to `application/vnd.kafka.json.v2+json` and the body format is `{ "records": [{ "value": { ... } }] }`. However, the Confluent API key/secret must go in a Basic Auth header, making the REST Proxy route suitable only if you are comfortable with that key living in your API Group — this is a higher risk than using a Cloud Function proxy. For production, always use the Cloud Function wrapper.

```
// Firebase Cloud Function — Kafka producer (index.js)
const functions = require('firebase-functions');
const { Kafka } = require('kafkajs');

const kafka = new Kafka({
  clientId: 'flutterflow-producer',
  brokers: [functions.config().kafka.broker],
  ssl: true,
  sasl: {
    mechanism: 'plain',
    username: functions.config().kafka.api_key,
    password: functions.config().kafka.api_secret,
  },
});

exports.produceKafkaMessage = functions.https.onRequest(async (req, res) => {
  res.set('Access-Control-Allow-Origin', '*');
  if (req.method === 'OPTIONS') {
    res.set('Access-Control-Allow-Methods', 'POST');
    res.set('Access-Control-Allow-Headers', 'Content-Type');
    return res.status(204).send('');
  }

  const { topic, key, value } = req.body;
  if (!topic || !value) {
    return res.status(400).json({ error: 'topic and value are required' });
  }

  const producer = kafka.producer();
  try {
    await producer.connect();
    await producer.send({
      topic,
      messages: [{ key: key || null, value: JSON.stringify(value) }],
    });
    await producer.disconnect();
    return res.status(200).json({ status: 'produced' });
  } catch (err) {
    console.error('Kafka produce error:', err);
    return res.status(500).json({ error: err.message });
  }
});
```

**Expected result:** The Cloud Function is deployed. A test POST with `{"topic":"app_events","value":{"event":"test"}}` returns `{"status":"produced"}` and the message appears in the Confluent Cloud topic viewer.

### 3. Build the FlutterFlow API Group to call the producer

With your Kafka producer endpoint ready, connect it to FlutterFlow. Click 'API Calls' in the left navigation, then '+ Add' → 'Create API Group'. Name it 'KafkaProducer'. Set the Base URL to your deployed Cloud Function's base URL (e.g. `https://us-central1-yourproject.cloudfunctions.net`).

Add a new API Call inside the group. Name it 'ProduceEvent'. Set the Method to POST and the URL to `/produceKafkaMessage`. Switch to the Body tab, set Body Type to JSON. Add variables for `topic` (String), `eventType` (String), and `payload` (JSON String) so you can pass dynamic values from the FlutterFlow Action Flow. Build the body template as `{ "topic": "{{topic}}", "value": { "type": "{{eventType}}", "data": {{payload}} } }`.

Test the call in the Response & Test tab with sample values. You should get a `{"status":"produced"}` response and be able to verify the message arrived in Confluent Cloud's topic viewer. Once tested, wire the call to an Action Flow: select the widget (e.g. a Submit button), open the Actions panel, click '+ Add Action' → 'Backend/API Call' → 'KafkaProducer → ProduceEvent', and map the variable values from your form fields or App State.

Note that the Cloud Function will have a cold-start latency of 1-5 seconds on the first call after inactivity. Subsequent calls to a warm function return in under 500ms. Consider showing a loading indicator in your FlutterFlow UI while the action completes.

```
{
  "method": "POST",
  "url": "https://us-central1-yourproject.cloudfunctions.net/produceKafkaMessage",
  "headers": {
    "Content-Type": "application/json"
  },
  "body": {
    "topic": "{{topic}}",
    "value": {
      "type": "{{eventType}}",
      "data": "{{payload}}"
    }
  }
}
```

**Expected result:** Tapping the Submit button in FlutterFlow successfully produces a Kafka message. The message appears in the Confluent Cloud topic viewer within a few seconds.

### 4. Set up a backend Kafka consumer that writes to Firestore or Supabase

For displaying Kafka events in FlutterFlow in real time, you need a persistent backend consumer. This cannot be a Cloud Function on its own (Functions have execution time limits and are not designed to hold a long-lived Kafka consumer group). Options include a Cloud Run service, a small Compute Engine VM, or a managed process on Render, Railway, or Heroku.

Write a Node.js consumer using `kafkajs`. It subscribes to your topic from the beginning (or a specific offset), processes each incoming message, and writes the relevant data into a Firestore collection or a Supabase table. Use Firestore's real-time update capability: each Kafka event creates or updates a document. Multiple FlutterFlow clients listening to that collection will all receive the update simultaneously via Firestore's WebSocket channel.

Structure the Firestore documents so they are easy to query from FlutterFlow. For example, for order status events, create a document per order ID at `order_events/{orderId}` with a `status` field and `updated_at` timestamp. FlutterFlow's Firebase integration can listen to that document in real time using the 'Document from reference' query type with a real-time updates toggle.

Deploy the consumer as a persistent service. Set Kafka credentials as environment variables in your hosting platform — never hardcode them. Monitor the consumer group lag in Confluent Cloud to ensure events are being processed promptly.

```
// Kafka consumer → Firestore bridge (Node.js)
const { Kafka } = require('kafkajs');
const admin = require('firebase-admin');

admin.initializeApp();
const db = admin.firestore();

const kafka = new Kafka({
  clientId: 'firestore-bridge',
  brokers: [process.env.KAFKA_BROKER],
  ssl: true,
  sasl: {
    mechanism: 'plain',
    username: process.env.KAFKA_API_KEY,
    password: process.env.KAFKA_API_SECRET,
  },
});

const consumer = kafka.consumer({ groupId: 'firestore-bridge-group' });

async function run() {
  await consumer.connect();
  await consumer.subscribe({ topic: 'order_events', fromBeginning: false });

  await consumer.run({
    eachMessage: async ({ message }) => {
      const event = JSON.parse(message.value.toString());
      // Write to Firestore so FlutterFlow can subscribe
      await db.collection('order_events').doc(event.orderId).set({
        status: event.status,
        updated_at: admin.firestore.FieldValue.serverTimestamp(),
      }, { merge: true });
    },
  });
}

run().catch(console.error);
```

**Expected result:** The consumer service is running. Producing a test message to the Kafka topic results in a new or updated document appearing in Firestore within 1-2 seconds.

### 5. Subscribe FlutterFlow widgets to the downstream Firestore or Supabase store

With the Kafka consumer writing events to Firestore (or Supabase Realtime), FlutterFlow subscribes to that store natively — it does not need to know Kafka exists at this point. This is the cleanest part of the architecture because you are leveraging FlutterFlow's built-in real-time database support.

For Firestore: In your FlutterFlow screen, add a Backend Query to a Column or ListView widget. Choose 'Firebase Query' → 'Collection' or 'Single Document', select the `order_events` collection, add a filter for the relevant document ID or field, and enable the 'Real-Time Updates' toggle. The widget will re-render every time the Kafka consumer writes a new event to Firestore — your users see live updates.

For Supabase Realtime: If your consumer writes to Supabase instead, use FlutterFlow's Supabase integration. Add a Backend Query, choose 'Supabase Query', select your table, add filters, and enable real-time. FlutterFlow will open a WebSocket subscription to Supabase and update the UI as rows change.

Style the screen to reflect the event data. For order status, use a Stepper widget or a series of Row widgets with conditional icons showing the current step. For sensor readings, use a LineChart widget with the latest values. If you'd rather have RapidDev's team wire up the full Kafka → Cloud Function → Firestore → FlutterFlow pipeline for you, we offer free scoping calls at rapidevelopers.com/contact.

**Expected result:** The FlutterFlow screen updates in real time as Kafka events arrive. Users see status changes or new readings without refreshing the screen.

## Best practices

- Never put Kafka API keys, SASL credentials, or broker addresses in FlutterFlow — all Kafka authentication must live in a Cloud Function or backend service config.
- Use the REST Proxy or a Cloud Function producer for outbound messages, and a persistent backend consumer feeding Firestore or Supabase Realtime for inbound — never attempt direct broker connections from Flutter.
- Set meaningful consumer group IDs (e.g. 'flutterflow-bridge-prod') and monitor consumer lag in Confluent Cloud so you know if your Firestore bridge falls behind.
- For high-frequency events, aggregate or downsample in the consumer before writing to Firestore or Supabase — writing every raw event at 10Hz will exhaust write quotas and cost more than necessary.
- Add a `source` field to every produced event payload (e.g. `"source": "flutterflow-app"`) to make consumer-side filtering and debugging easier.
- Test the full pipeline end-to-end with a single test message before building the FlutterFlow UI — verify that producing a message appears in Confluent Cloud, then in Firestore, before wiring up the real-time subscription.
- Use `fromBeginning: false` in the consumer to avoid replaying historical events on restart — only process new messages unless your use-case specifically needs historical replay.
- Deploy the Kafka consumer as a long-running service (Cloud Run always-on, Compute Engine, or similar) rather than a cron-triggered Cloud Function — Kafka consumers need a persistent connection to maintain their consumer group membership and offset commits.

## Use cases

### Order tracking app that displays real-time order status events from Kafka

An e-commerce founder's backend publishes order-status events (placed, packed, shipped, delivered) to a Kafka topic. A backend consumer service reads these events and writes the latest status for each order into a Firestore collection. A FlutterFlow app subscribes to that Firestore collection via the native Firebase integration and shows users live order status without polling. Kafka handles the high-throughput pipeline; FlutterFlow just reads from Firestore.

Prompt example:

```
Build an order status screen that listens to a Firestore 'order_events' collection filtered by the current user's order ID. Show a timeline of status changes (placed, packed, shipped, delivered) with timestamps as events arrive in real time.
```

### IoT sensor dashboard with live readings from a Kafka stream

A hardware startup streams sensor readings (temperature, humidity, pressure) from devices to a Kafka topic at high frequency. A backend consumer aggregates and downsamples the readings every 5 seconds and writes them to Supabase Realtime. A FlutterFlow dashboard subscribes to Supabase and renders live-updating charts. The FlutterFlow app never talks to Kafka — it only reads from Supabase's HTTPS WebSocket channel.

Prompt example:

```
Create a sensor dashboard screen with line charts for temperature and humidity. Subscribe to a Supabase Realtime channel called 'sensor_readings' and update the charts as new rows arrive. Show the last-updated timestamp.
```

### User activity logging that produces events to a Kafka topic

A product analytics team wants to capture in-app events (button taps, screen views, purchases) and route them into a Kafka topic for their data pipeline. A Firebase Cloud Function wraps a Kafka producer and accepts POST requests from FlutterFlow with an event type and properties payload. The FlutterFlow Action Flow fires this call after key user actions. Kafka credentials stay in Firebase Function config.

Prompt example:

```
After a user completes a purchase, trigger an API call to a Cloud Function that produces a 'purchase_completed' event to a Kafka topic named 'app_events' with fields: user_id, product_id, and amount.
```

## Troubleshooting

### Trying to connect to Kafka brokers from a Custom Action throws 'Unsupported operation: Socket' or the connection simply hangs

Cause: Kafka's native protocol requires TCP connections on port 9092. Flutter web builds cannot open TCP sockets, and even on mobile, there is no production-grade Dart Kafka client that is cross-platform. Custom Action Dart code cannot speak the Kafka wire protocol.

Solution: Do not attempt a direct broker connection from FlutterFlow. Use the REST Proxy or Cloud Function producer pattern described in this guide. The only correct architecture is HTTPS from FlutterFlow to an HTTP bridge, not TCP from FlutterFlow to brokers.

### Confluent REST Proxy returns 415 Unsupported Media Type when producing messages

Cause: The `Content-Type` header is missing or set to `application/json` instead of the Confluent-specific media type required by the REST Proxy.

Solution: Set the `Content-Type` header to exactly `application/vnd.kafka.json.v2+json` in your FlutterFlow API Call headers. The REST Proxy requires this specific content type to parse the JSON records array correctly.

### The Cloud Function producer returns 500 — 'Kafka broker not reachable' or 'SASL authentication failed'

Cause: Either the Confluent Cloud broker URL is malformed (missing port or wrong hostname), the API key/secret is incorrect, or the Firebase Function config values were not set or deployed.

Solution: Verify your broker URL format: `pkc-xxxxx.region.provider.confluent.cloud:9092`. Check that the API key and secret in Firebase Function config match the values in Confluent Cloud → API Keys. Redeploy the function after updating config values with `firebase deploy --only functions`. Check Function logs in the Firebase console for the specific error.

### The backend consumer is running but Firestore documents are not updating when Kafka messages arrive

Cause: The consumer may have lost its Kafka connection, the Firestore service account key may be missing or revoked, or the consumer is in an error loop and not processing messages.

Solution: Check the consumer service logs (in Cloud Run, Compute Engine, or your hosting platform's log viewer). Verify the consumer group is showing up in Confluent Cloud → Consumer Groups with non-growing lag. Ensure the Firebase service account key used by the consumer has Firestore write permissions. Add explicit error logging in the `eachMessage` handler to surface write failures.

## Frequently asked questions

### Can FlutterFlow subscribe to a Kafka topic directly and show events in a list in real time?

Not directly — Kafka's protocol is TCP-based and a Flutter app cannot speak it. The pattern is to run a backend consumer that reads from Kafka and writes events into Firestore or Supabase Realtime, then subscribe FlutterFlow to that downstream store. From the user's perspective the experience is real-time; under the hood there is a 1-3 second bridge latency.

### What is the simplest way to produce a Kafka message from a FlutterFlow button tap?

Write a Firebase Cloud Function that accepts a POST request with a topic and a payload, uses kafkajs to produce the message, and returns a success response. Create a FlutterFlow API Call pointing at that Function URL. Add a 'Backend/API Call' action to your button's Action Flow and pass the payload as variables. The function handles all Kafka authentication server-side.

### Do I need Confluent Cloud or can I use a self-hosted Kafka cluster?

Both work. For Confluent Cloud, the REST Proxy is built in and your Cloud Function just needs the bootstrap URL, API key, and secret. For a self-hosted cluster, you can either enable the Confluent REST Proxy on your cluster or connect directly from your Cloud Function using kafkajs with your broker's hostname and SASL config. The FlutterFlow side is identical either way.

### How much will this architecture cost on Confluent Cloud's free tier?

Confluent Cloud offers approximately $400 in free credits for new accounts — check confluent.io for current terms. After credits are used, you pay for usage (data in/out, partitions, retention). For typical FlutterFlow apps that produce a few hundred events per day, the monthly cost is low. The Firebase Cloud Function side is free for the first 2 million invocations per month.

### What happens to unconsumed Kafka events if my backend consumer goes down?

Kafka retains events for the configured retention period (default 7 days on most clusters). When your consumer comes back online, it resumes from the last committed offset, so no events are lost. However, there will be a gap in your Firestore or Supabase mirror during the downtime — users may see stale data until the consumer catches up. For critical apps, run your consumer with redundancy (multiple replicas) and health checks.

---

Source: https://www.rapidevelopers.com/flutterflow-integrations/apache-kafka
© RapidDev — https://www.rapidevelopers.com/flutterflow-integrations/apache-kafka