# How to Monitor Success Rates of Language Model Calls in n8n

- Tool: n8n
- Difficulty: Advanced
- Time required: 40-50 minutes
- Compatibility: n8n 1.30+, PostgreSQL or MySQL, any LLM provider (OpenAI, Claude, Mistral, Cohere)
- Last updated: March 2026

## TL;DR

Track LLM call reliability in n8n by building a monitoring workflow that logs every API call's success, failure, latency, and token usage to a database. Use the Error Trigger node to capture failures, a Code node to compute metrics, and a scheduled aggregation workflow to generate daily success rate reports and alerts when reliability drops below thresholds.

## Why You Need to Monitor LLM Call Success Rates

LLM APIs are inherently unreliable — they experience rate limits, timeouts, server errors, and content filtering blocks. Without monitoring, you have no visibility into how often your n8n workflows succeed or fail, how much latency users experience, or how much you are spending on API calls. This tutorial shows how to build a complete monitoring system inside n8n itself, using error workflows, logging nodes, and a daily aggregation workflow that alerts you when success rates drop below acceptable thresholds.

## Before you start

- A running n8n instance (self-hosted or cloud) on version 1.30 or later
- A PostgreSQL or MySQL database accessible from n8n
- At least one workflow that calls an LLM API (OpenAI, Claude, Mistral, etc.)
- Basic understanding of SQL INSERT and SELECT queries
- An email or Slack credential for alert notifications

## Step-by-step guide

### 1. Create the logging database table

Create a PostgreSQL table to store LLM call logs. Each row represents one API call with its outcome, latency, token usage, and error details. Use the Postgres node in a setup workflow or run the SQL directly in your database client. The table schema captures everything needed for success rate calculations, cost tracking, and failure analysis.

```
-- PostgreSQL: Create the LLM call log table
CREATE TABLE IF NOT EXISTS llm_call_logs (
  id SERIAL PRIMARY KEY,
  workflow_id VARCHAR(255) NOT NULL,
  workflow_name VARCHAR(255),
  node_name VARCHAR(255),
  provider VARCHAR(50) NOT NULL,       -- openai, anthropic, mistral, cohere
  model VARCHAR(100),
  status VARCHAR(20) NOT NULL,          -- success, error, timeout, rate_limited
  error_message TEXT,
  error_code VARCHAR(10),
  latency_ms INTEGER,
  prompt_tokens INTEGER,
  completion_tokens INTEGER,
  total_tokens INTEGER,
  session_id VARCHAR(255),
  created_at TIMESTAMP DEFAULT NOW()
);

CREATE INDEX idx_llm_logs_created ON llm_call_logs(created_at);
CREATE INDEX idx_llm_logs_status ON llm_call_logs(status);
CREATE INDEX idx_llm_logs_provider ON llm_call_logs(provider);
```

**Expected result:** The llm_call_logs table exists in your PostgreSQL database with indexes for efficient querying.

### 2. Add success logging after each LLM node

After each LLM API call in your workflow (OpenAI node, HTTP Request to Claude, etc.), add a Code node followed by a Postgres node to log the successful call. The Code node extracts the relevant metrics (latency, tokens, model) from the API response, and the Postgres node inserts them into the log table. Connect this logging branch to the success output of your LLM node so it only fires on successful calls.

```
// Code node — JavaScript
// Extract metrics from a successful LLM call
// Place after your OpenAI/Claude/Mistral node

const item = $input.first();
const startTime = $('Webhook').first().json._startTime || Date.now();
const latencyMs = Date.now() - startTime;

// Extract token usage (format varies by provider)
const usage = item.json.usage || item.json.message?.usage || {};

return [{
  json: {
    workflow_id: $workflow.id,
    workflow_name: $workflow.name,
    node_name: $node.name,
    provider: 'openai', // Change per provider
    model: item.json.model || 'unknown',
    status: 'success',
    error_message: null,
    error_code: null,
    latency_ms: latencyMs,
    prompt_tokens: usage.prompt_tokens || usage.input_tokens || 0,
    completion_tokens: usage.completion_tokens || usage.output_tokens || 0,
    total_tokens: (usage.prompt_tokens || usage.input_tokens || 0) + (usage.completion_tokens || usage.output_tokens || 0),
    session_id: $('Webhook').first().json.sessionId || null
  }
}];
```

**Expected result:** Every successful LLM call is logged to PostgreSQL with latency, token usage, and model information.

### 3. Set up the Error Trigger workflow for failure logging

Create a separate Error Workflow that is triggered whenever any LLM workflow fails. Go to Workflow Settings in each LLM workflow and set the Error Workflow to this new workflow. The Error Trigger node receives the full error details including the failed node name, error message, and execution ID. Parse the error to categorize it (timeout, rate_limit, auth_error, server_error) and log it to the same database table.

```
// Code node in Error Workflow — JavaScript
// Categorize and log LLM failures

const errorData = $input.first().json;
const errorMessage = errorData.execution?.error?.message || 'Unknown error';
const nodeName = errorData.execution?.lastNodeExecuted || 'Unknown';

// Categorize the error
let status = 'error';
let errorCode = '';

if (errorMessage.includes('ETIMEDOUT') || errorMessage.includes('timeout')) {
  status = 'timeout';
  errorCode = '408';
} else if (errorMessage.includes('429') || errorMessage.includes('rate limit')) {
  status = 'rate_limited';
  errorCode = '429';
} else if (errorMessage.includes('401') || errorMessage.includes('Unauthorized')) {
  status = 'auth_error';
  errorCode = '401';
} else if (errorMessage.includes('500') || errorMessage.includes('Internal Server')) {
  status = 'server_error';
  errorCode = '500';
} else if (errorMessage.includes('529') || errorMessage.includes('overloaded')) {
  status = 'server_error';
  errorCode = '529';
}

return [{
  json: {
    workflow_id: errorData.workflow?.id || 'unknown',
    workflow_name: errorData.workflow?.name || 'unknown',
    node_name: nodeName,
    provider: nodeName.toLowerCase().includes('openai') ? 'openai'
      : nodeName.toLowerCase().includes('claude') ? 'anthropic'
      : nodeName.toLowerCase().includes('mistral') ? 'mistral'
      : 'unknown',
    model: 'unknown',
    status,
    error_message: errorMessage.substring(0, 500),
    error_code: errorCode,
    latency_ms: null,
    prompt_tokens: 0,
    completion_tokens: 0,
    total_tokens: 0,
    session_id: null
  }
}];
```

**Expected result:** Every LLM workflow failure is automatically categorized and logged to the same llm_call_logs table.

### 4. Build the daily aggregation and alerting workflow

Create a scheduled workflow that runs daily (using the Schedule Trigger node set to run at 9:00 AM). It queries the llm_call_logs table for the last 24 hours, computes success rates per provider and per workflow, and sends an alert via email or Slack if any success rate drops below 95%. This gives you a daily reliability report without needing external monitoring tools.

```
-- SQL query for the Postgres node
-- Daily success rate aggregation
SELECT
  provider,
  COUNT(*) as total_calls,
  COUNT(*) FILTER (WHERE status = 'success') as successful_calls,
  ROUND(
    COUNT(*) FILTER (WHERE status = 'success')::numeric / COUNT(*)::numeric * 100, 2
  ) as success_rate,
  COUNT(*) FILTER (WHERE status = 'timeout') as timeouts,
  COUNT(*) FILTER (WHERE status = 'rate_limited') as rate_limits,
  COUNT(*) FILTER (WHERE status = 'error') as errors,
  ROUND(AVG(latency_ms) FILTER (WHERE status = 'success'), 0) as avg_latency_ms,
  SUM(total_tokens) as total_tokens_used
FROM llm_call_logs
WHERE created_at >= NOW() - INTERVAL '24 hours'
GROUP BY provider
ORDER BY success_rate ASC;
```

**Expected result:** A daily report showing success rate, average latency, and token usage per LLM provider, with alerts for low reliability.

### 5. Add alerting logic for low success rates

After the aggregation query, add a Code node that checks if any provider's success rate is below your threshold (e.g., 95%). If so, format an alert message and route it to the Send Email or Slack node. Use the IF node to branch: if all rates are above threshold, skip the alert. This prevents alert fatigue while ensuring you are notified of real problems.

```
// Code node — JavaScript
// Check success rates and generate alert if needed

const items = $input.all();
const THRESHOLD = 95; // Alert if success rate drops below 95%

const alerts = [];
const summary = [];

for (const item of items) {
  const rate = parseFloat(item.json.success_rate);
  const provider = item.json.provider;

  summary.push(
    `${provider}: ${rate}% success (${item.json.total_calls} calls, ${item.json.avg_latency_ms}ms avg latency)`
  );

  if (rate < THRESHOLD) {
    alerts.push(
      `⚠ ${provider}: ${rate}% success rate (${item.json.errors} errors, ${item.json.timeouts} timeouts, ${item.json.rate_limits} rate limits)`
    );
  }
}

return [{
  json: {
    hasAlerts: alerts.length > 0,
    alertMessage: alerts.length > 0
      ? `LLM Success Rate Alert\n\n${alerts.join('\n')}\n\nFull Summary:\n${summary.join('\n')}`
      : null,
    summaryMessage: `Daily LLM Report\n\n${summary.join('\n')}`,
    alertCount: alerts.length
  }
}];
```

**Expected result:** An alert is generated only when a provider's success rate drops below 95%, with a full summary of all providers included.

## Complete code example

File: `llm-success-logger.js`

```javascript
// Complete Code node: Universal LLM call logger
// Place after any LLM node. Works with OpenAI, Claude, Mistral, Cohere.

const item = $input.first();
const response = item.json;

// Detect provider from response structure
let provider = 'unknown';
let model = 'unknown';
let promptTokens = 0;
let completionTokens = 0;

if (response.model?.includes('gpt') || response.model?.includes('o1')) {
  provider = 'openai';
  model = response.model;
  promptTokens = response.usage?.prompt_tokens || 0;
  completionTokens = response.usage?.completion_tokens || 0;
} else if (response.model?.includes('claude')) {
  provider = 'anthropic';
  model = response.model;
  promptTokens = response.usage?.input_tokens || 0;
  completionTokens = response.usage?.output_tokens || 0;
} else if (response.model?.includes('mistral')) {
  provider = 'mistral';
  model = response.model;
  promptTokens = response.usage?.prompt_tokens || 0;
  completionTokens = response.usage?.completion_tokens || 0;
} else if (response.generation_id) {
  provider = 'cohere';
  model = response.model || 'command';
  promptTokens = response.meta?.billed_units?.input_tokens || 0;
  completionTokens = response.meta?.billed_units?.output_tokens || 0;
}

// Calculate latency if start time was recorded
const startTime = item.json._startTime || $('Set Start Time')?.first()?.json?._startTime;
const latencyMs = startTime ? Date.now() - startTime : null;

return [{
  json: {
    workflow_id: $workflow.id,
    workflow_name: $workflow.name,
    node_name: $node.name,
    provider,
    model,
    status: 'success',
    error_message: null,
    error_code: null,
    latency_ms: latencyMs,
    prompt_tokens: promptTokens,
    completion_tokens: completionTokens,
    total_tokens: promptTokens + completionTokens,
    session_id: null,
    created_at: new Date().toISOString()
  }
}];
```

## Common mistakes

- **Only logging failures without logging successes, making success rate calculation impossible** — undefined Fix: Log every LLM call — successes and failures — to get accurate denominators for success rate percentages
- **Putting the logging Postgres node in the main execution path, where a DB failure blocks the LLM response** — undefined Fix: Use the 'Continue On Fail' setting on the logging node so database issues do not block the main workflow
- **Not setting the Error Workflow in LLM workflow settings, missing failures that crash the workflow** — undefined Fix: Go to each LLM workflow's Settings and set the Error Workflow to your error logging workflow
- **Alerting on every single failure instead of aggregating to success rates** — undefined Fix: Use the daily aggregation workflow to check rates — individual failures are normal and expected
- **Not categorizing errors (timeout vs rate limit vs auth), making root cause analysis difficult** — undefined Fix: Parse error messages in the Error Workflow Code node and assign status categories (timeout, rate_limited, auth_error)

## Best practices

- Log both successes and failures to the same table for accurate success rate calculations
- Use the Error Trigger node for failure logging — it captures errors that crash the workflow
- Set a 95% success rate threshold for alerts — below this indicates a systemic issue
- Track latency separately from success/failure — high latency is a leading indicator of future failures
- Index the created_at and status columns for fast aggregation queries
- Run the aggregation workflow daily, not hourly — hourly alerts cause fatigue without actionable insight
- Include token usage in logs to track API costs alongside reliability
- Retain logs for at least 90 days to identify long-term trends

## Frequently asked questions

### Can I use n8n's built-in execution history instead of a custom logging table?

n8n's execution history shows workflow-level results but does not aggregate metrics like success rates, average latency, or token usage. Custom logging provides the structured data needed for monitoring and alerting.

### How much storage does the logging table use?

Each log row is approximately 500 bytes. At 1,000 LLM calls per day, that is about 500KB/day or 15MB/month — negligible for any PostgreSQL instance.

### Should I log LLM calls from test executions?

Add a flag to exclude test executions from production metrics. Check if the execution was triggered manually (test) or by a trigger (production) and filter accordingly in your aggregation queries.

### What is a normal success rate for LLM API calls?

A well-configured workflow should achieve 97-99% success rates. Rates below 95% indicate a systemic issue (wrong model, insufficient retries, rate limit misconfiguration). Rates below 90% require immediate investigation.

### Can I send monitoring data to external tools like Grafana or Datadog?

Yes, you can extend the logging workflow to send metrics to external monitoring tools via HTTP Request nodes. Send data to Grafana Cloud's push API, Datadog's metrics endpoint, or any other monitoring service.

### Can RapidDev help build a comprehensive monitoring system for n8n LLM workflows?

Yes, RapidDev builds enterprise-grade monitoring for n8n including LLM call tracking, cost analytics, latency dashboards, and alerting. Their team can integrate monitoring with Grafana, Datadog, or custom dashboards tailored to your workflow architecture.

---

Source: https://www.rapidevelopers.com/n8n-tutorial/how-to-monitor-success-rates-of-language-model-calls-in-n8n
© RapidDev — https://www.rapidevelopers.com/n8n-tutorial/how-to-monitor-success-rates-of-language-model-calls-in-n8n