How to Handle Large JSON in n8n

The SplitInBatches node divides a large array of items into smaller batches and processes each batch one at a time. This prevents n8n from loading the entire dataset into memory at once. Add a SplitInBatches node after the node that produces the large dataset, set the batch size (e.g., 50 or 100 items), then connect your processing nodes to the SplitInBatches output. The SplitInBatches node loops automatically until all items are processed.

Large API responses often include fields you do not need. Use the Edit Fields (formerly Set) node to keep only the fields required by downstream nodes. This dramatically reduces memory usage when dealing with API responses that return hundreds of fields per item. Place the Edit Fields node right after the data source node, before any heavy processing.

1// Example: API returns 50 fields per item, but you only need 3
2// In the Edit Fields node, set mode to 'Manual Mapping' and add:
3//   name  → {{ $json.name }}
4//   email → {{ $json.email }}
5//   id    → {{ $json.id }}
6// Enable 'Keep Only Set' to discard all other fields
7
8// Alternatively, use a Code node:
9const items = $input.all();
10return items.map(item => ({
11  json: {
12    id: item.json.id,
13    name: item.json.name,
14    email: item.json.email
15  }
16}));

By default, Node.js limits memory to about 1.5 GB. For workflows that genuinely need to handle large datasets in memory, you can increase this limit. Set the NODE_OPTIONS environment variable before starting n8n. For Docker, add it to your compose file or run command.

1# Increase memory limit to 4 GB
2export NODE_OPTIONS="--max-old-space-size=4096"
3n8n start
4
5# Docker run
6docker run -d \
7  --name n8n \
8  -e NODE_OPTIONS="--max-old-space-size=4096" \
9  -p 5678:5678 \
10  docker.n8n.io/n8nio/n8n
11
12# docker-compose.yml
13# environment:
14#   - NODE_OPTIONS=--max-old-space-size=4096

Instead of requesting all data at once, use pagination to fetch data in pages. Most APIs support limit and offset parameters. Use a Loop node or the HTTP Request node's built-in pagination feature to fetch one page at a time, process it, and then fetch the next page. This keeps memory usage constant regardless of total dataset size.

1// HTTP Request node pagination settings:
2// Pagination Mode: Response Contains Next URL
3// or
4// Pagination Mode: Offset-Based
5//   Limit Parameter: limit
6//   Offset Parameter: offset
7//   Page Size: 100
8//   Max Pages: 50

n8n saves the input and output of every node for each execution by default. For workflows that process large datasets, this execution data can be enormous and slow down the database. In the workflow settings, disable Save Successful Execution Data or set it to save only the last N executions. This prevents the execution history from consuming disk space and memory.