Skip to content
FrAIFlow

Solutions

Fix Extra data by recognizing JSON Lines

When a parser accepts the first object and fails at the next, inspect the container format before editing the data. A single JSON document has one top-level value. JSON Lines stores one complete JSON value on each line. Two valid objects together are not necessarily one valid document.

A reported JSONL-to-CSV conversion failed on line two. In another public discussion, a script wrote serialized objects together; the reporter confirmed that adding a newline after each object fixed the intended line-oriented output. These reports do not mean every Extra data error is JSON Lines.

Choose the reader that matches your file

Open a copy in a text editor and look at the first few records. If each complete object is on its own physical line, you may have JSON Lines. If one object spans several lines, this recipe is not the right reader. If objects are joined without any separator, fix the writer or use a parser designed for that stream; splitting on closing braces will break nested objects and quoted text.

For a small JSON Lines file, this JavaScript helper keeps the line number when parsing fails. It allows one final newline but does not quietly throw away blank records.

function readJsonLines(text) {
  if (text === "") return [];
  const lines = text.split(/\r?\n/);
  if (lines.at(-1) === "") lines.pop();
  return lines.map((line, i) => {
    if (!line.trim()) throw new Error(`Blank record on line ${i + 1}`);
    try { return JSON.parse(line); }
    catch { throw new Error(`Invalid JSON on line ${i + 1}`); }
  });
}
const source = '{"id":"001"}\n{"id":"002"}\n';
const records = readJsonLines(source);
console.log(records.length); // 2
const ordinaryJson = JSON.stringify(records, null, 2);
const jsonLines = records.map(item => JSON.stringify(item)).join("\n") + "\n";

Choose ordinaryJson when the receiving program expects one array document. Choose jsonLines when it expects one record per line. Do not add commas between the latter records. For an empty record set, write an empty JSON Lines file rather than the single newline produced by the last example expression.

Check the number of parsed records against the intended source count and inspect the first and last IDs. If a line fails, inspect that line in the original copy. Skipping it may make a conversion “finish” while silently dropping an order or a log event. A reported line number is safer than logging the entire record if it contains private information.

When the file is too large to fit comfortably in memory, replace this small-file helper with a streaming line reader. Preserve the same error and record-count checks. JSON Lines solves record boundaries; it does not flatten nested values into CSV columns for you.

Record boundaries are part of the format

The helper accepts LF and CRLF endings, not bare CR. A .json extension alone does not establish the record format. If CSV is your destination, decide how missing and nested fields map to columns, then use a CSV writer to handle commas, quotes and embedded newlines.

Two records / {"id":"001"} / {"id":"002"}
Author-created explanation, not an application capture.

Our Node v24.19.0 fixture used {"id":"001"} and {"id":"002"} on separate lines. Parsing the whole input failed at the second object; parsing the two lines yielded two records, and JSON.stringify(records) produced a valid array document. Error wording and positions vary by runtime.

Choose the format for your output file

Python’s documentation warns that repeated dump calls do not create a valid single JSON document: JSON is not a framed protocol. A .json filename is not enough to establish the framing contract. Confirm the expected format with the tool that will read it.

Two JSON Lines records with IDs 001 and 002 compared with a JSON array containing the same records.
Supporting explanation, not an application screenshot.
Recorded Node.js output: parsing the input as one document fails, while parsing each line preserves both records.
Saved local test output. Open the image to inspect the original evidence.
Sources, examples and further reading