lesson 225 Data Formats & Analytical Workflows · advanced

Exporting results

A result isn't finished until someone else can read it back exactly. Write CSV without the index and with a fixed number format, JSON with ISO dates, and check the file by reading it back.

You'll be able to

Write CSV with index=False and float_format="%.2f", and explain the Unnamed: 0 column
Write JSON records and JSON Lines with to_json(orient="records", lines=..., date_format="iso")
Read an export back (read_csv, json.loads) and compare it with what was meant

The idea

Exports are read by other programs and other people. to_csv(path) writes the index as a first, unnamed column unless you pass index=False; read back, it turns into a column called Unnamed: 0. Floats are written with every digit unless you choose a format: float_format="%.2f" writes money as cents. to_json(path, orient="records") writes a list of objects, one per row; with lines=True it writes one JSON object per line (JSON Lines, the CSV JSON & JSONL module), which streaming tools prefer. date_format="iso" writes timestamps as ISO 8601 text, the format every language can parse.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · CSV that reads back cleanly

import pandas as pd

summary = pd.DataFrame({"borough": ["Manhattan", "Queens"], "trips": [9, 1], "revenue": [340.4, 66.71]})
summary.to_csv("with_index.csv")
summary.to_csv("summary.csv", index=False, float_format="%.2f")
print(pd.read_csv("with_index.csv").columns.tolist())
with open("summary.csv", encoding="utf-8") as f:
    print(f.read(), end="")
print(pd.read_csv("summary.csv").equals(summary))

example-2.py · Reading JSON Lines as one JSON document

import pandas as pd

trips = pd.DataFrame({"trip_id": [16126, 16127], "fare_amount": [40.8, 7.2]})
trips.to_json("trips.jsonl", orient="records", lines=True)
pd.read_json("trips.jsonl")

example-3.py · Trips as JSON Lines with ISO timestamps

import json
import pandas as pd

trips = pd.DataFrame({
    "trip_id": [16126, 16127],
    "pickup_at": pd.to_datetime(["2025-01-15 00:12:47", "2025-01-15 00:18:28"]),
    "tip_amount": [10.7, None],
})
trips.to_json("trips.jsonl", orient="records", lines=True, date_format="iso")
with open("trips.jsonl", encoding="utf-8") as f:
    lines = f.read().splitlines()
print(lines[0])
print(json.loads(lines[1]))
print(pd.read_json("trips.jsonl", lines=True)["trip_id"].tolist())

When to use which: CSV for tables people open in spreadsheets; JSON records for web APIs and nested consumers; JSON Lines for logs and large streams read one record at a time; Parquet (“Parquet files”) for typed data other programs reread. Mistakes to avoid: exporting the index by accident; floats like 22.459999999999997 in a report; dates in a local format nobody else parses; and never reading your own export back to check it.

Review

to_csv(index=False, float_format="%.2f") writes clean, rounded CSV; a written index comes back as Unnamed: 0.
to_json(orient="records", lines=True, date_format="iso") writes JSON Lines with ISO timestamps; read it with lines=True.
Read every export back and compare it with what you meant to write.
Next (see Reliable Data Pipelines): pipelines that validate their input, can be rerun safely, and resume after a failure.

Practise this

Check a CSV export before sending it