lesson 98 CSV JSON & JSONL · intermediate

Parse JSON into Python values

JSON is how APIs and config files share nested data. `json.loads` (text) and `json.load` (a file) turn it into dictionaries, lists, numbers, text, booleans and None, and `JSONDecodeError` says exactly where invalid JSON goes wrong.

You'll be able to

Say which Python type each JSON value becomes (true → True, null → None, objects → dictionaries)
Read a JSON file with json.load and reach nested values with keys and get
Catch json.JSONDecodeError for text that isn't valid JSON, and read where the error is

The idea

CSV is a flat table of text. Real data is often nested: an event has an actor, the actor has a login; an API page has a list of features, each with properties. JSON writes that structure as text, and almost every web API and many config files use it.

import json, then json.loads(text) parses a string and json.load(file) parses an open file. JSON objects become dictionaries, arrays become lists, strings become str, numbers become int or float, true/false become True/False, and null becomes None.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · One repository event

import json

text = '{"id": "48000000581", "type": "PushEvent", "public": true, "payload": {"size": 2, "ref": null}}'
event = json.loads(text)
print(event["type"], event["public"], event["payload"]["ref"])
print(event["payload"]["size"] + 1)

JSON is stricter than Python. Keys and strings need double quotes, there are no trailing commas, and no comments. Invalid text raises json.JSONDecodeError, which is a kind of ValueError; its message gives the line and column where parsing stopped.

example-2.py · Single quotes are not JSON

import json

settings = json.loads("{'currency': 'USD'}")

API responses are usually saved as files. This function reads one page of the USGS earthquake feed with json.load and finds the strongest quake. Some events have no magnitude yet (null), so they must be skipped rather than compared.

example-3.py · The strongest earthquake on an API page

import json

def strongest(path):
    with open(path, encoding="utf-8") as source:
        page = json.load(source)
    best_place = None
    best_mag = None
    for feature in page["features"]:
        props = feature["properties"]
        if props.get("mag") is None:
            continue
        if best_mag is None or props["mag"] > best_mag:
            best_place = props["place"]
            best_mag = props["mag"]
    return (best_place, best_mag)

with open("page-001.json", "w", encoding="utf-8") as out:
    out.write("""{"type": "FeatureCollection", "metadata": {"count": 3},
 "features": [
  {"id": "us6000pw77", "properties": {"mag": 4.7, "place": "Arctic Ocean"}},
  {"id": "us6000pw78", "properties": {"mag": null, "place": "Fira, Greece"}},
  {"id": "us6000pw7k", "properties": {"mag": 5.1, "place": "Kermadec Islands"}}
 ]}""")
print(strongest("page-001.json"))

When to use it: whenever data arrives from an API, a config file or another program as JSON. Mistakes to avoid: treating a Python repr as JSON, assuming every key is present (use get for optional ones), and catching every exception when you mean json.JSONDecodeError.

Review

json.loads(text) and json.load(file) turn JSON into dictionaries, lists, numbers, text, booleans and None.
JSON needs double quotes and no trailing commas; invalid text raises json.JSONDecodeError (a ValueError).
Use get for optional keys and isinstance when the shape of the data isn't guaranteed.
Next: write Python values out as JSON.

Practise this

Validate a service's JSON settings