How to Queue OpenAI Requests When Too Many Users Hit the Workflow at Once

Before building the workflow, configure n8n itself to limit how many workflow executions run at once. Set the environment variable N8N_CONCURRENCY_PRODUCTION_LIMIT to a value that matches your OpenAI tier. For example, if your OpenAI tier allows 60 RPM, set this to 5 so at most 5 executions run simultaneously. This is your first line of defense against rate limit errors.

1# In your n8n environment configuration:
2N8N_CONCURRENCY_PRODUCTION_LIMIT=5

Add a Webhook node with HTTP Method POST and Path /ai-chat. Set the Response Mode to 'Using Respond to Webhook Node' so the caller waits for the actual AI response. This is critical because it keeps the HTTP connection open while the request waits in the queue, ensuring the user eventually gets their response even if it takes longer due to queuing.

When a single execution handles multiple items (for example, from a batch endpoint or a database poll), the SplitInBatches node processes them in groups. Set the Batch Size to 1 for strict sequential processing, or increase it to 3-5 if your rate limits allow small bursts. Connect the Webhook output to SplitInBatches. Each batch will go through the OpenAI call, then loop back for the next batch.

1// SplitInBatches configuration:
2// Batch Size: 1 (safest for rate limits)
3// Options > Reset: false (keep processing state)

Connect the first output of SplitInBatches (the batch output) to an OpenAI node. Configure it with your model (e.g., gpt-4o), the user prompt from {{ $json.message }}, and a system message. Keep max_tokens reasonable to stay within TPM limits. The SplitInBatches loop ensures only one call happens at a time.

Connect the OpenAI node output to a Wait node, then connect the Wait node back to the SplitInBatches node (the second input, which is the loop input). Set the Wait node to wait for 1-2 seconds. This creates a deliberate pause between each API call, preventing burst patterns that trigger rate limits even when you are technically under the RPM cap.

1// Wait node configuration:
2// Resume: After Time Interval
3// Wait Amount: 1
4// Wait Unit: Seconds

Even with queuing, edge cases can trigger 429 errors. Add an Error Trigger workflow that catches failed executions, waits 30-60 seconds using a Wait node, then retries the original request by calling the same webhook. In the main workflow, enable Settings > Error Workflow and point it to this retry workflow. Set a maximum retry count (stored in the payload) to prevent infinite loops.

1// In the Error Trigger workflow's Code node:
2const errorData = $input.first().json;
3const retryCount = (errorData.retryCount || 0) + 1;
4
5if (retryCount > 3) {
6  // Stop retrying after 3 attempts
7  return [{ json: { error: 'Max retries exceeded', originalMessage: errorData.message } }];
8}
9
10// Pass to Wait node (30s) then HTTP Request node to re-call webhook
11return [{ json: { ...errorData, retryCount } }];