lesson 84 Nested Data Structures · intermediate

Flatten nested records into rows

Spreadsheets, CSV files and most charts want flat rows. Flattening lifts nested values up into one row per record, or one row per inner item.

You'll be able to

Flatten a list of lists into one list with an inner loop
Build one flat dictionary per record by lifting values out of its nested parts
Produce one row per inner item, copying the outer record's fields onto each row

The idea

A GeoJSON feature is great for an API and awkward for a spreadsheet: the magnitude is two keys deep and the latitude is a position inside a list. Tables, CSV files and charts want flat rows: one dictionary per row, every value directly under a plain key.

Flattening builds those rows. The simplest form turns a list of lists into one list: an inner loop appends every inner value to a single result.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · Sessions per week into one list

weekly = [[3, 5, 2], [4], [], [6, 1]]
sessions = []
for week in weekly:
    for count in week:
        sessions.append(count)
print(sessions)
print(len(weekly), "weeks,", len(sessions), "sessions")

For records, flattening means building a new dictionary per record and lifting each nested value up to a plain key. Choose clear names for the new keys: a reader of the CSV won't know that latitude came from position 1 of coordinates.

example-2.py · Earthquakes as table rows

def to_rows(features):
    rows = []
    for feature in features:
        longitude, latitude, depth = feature["geometry"]["coordinates"]
        rows.append({
            "id": feature["id"],
            "mag": feature["properties"]["mag"],
            "place": feature["properties"]["place"],
            "latitude": latitude,
            "longitude": longitude,
            "depth_km": depth,
        })
    return rows

features = [
    {"id": "us6000pw77", "properties": {"mag": 4.7, "place": "Arctic Ocean"},
     "geometry": {"coordinates": [-107.5679, 80.7069, 10]}},
    {"id": "ak0252r9gt13", "properties": {"mag": 2.7, "place": "Southern Alaska"},
     "geometry": {"coordinates": [-147.2371, 61.4029, 9.6]}},
]
for row in to_rows(features):
    print(row)

When a record holds a list (an order's items, a question's tags), one row per record isn't enough. The usual flat shape is one row per inner item, with the outer record's fields copied onto every row so each row stands on its own. A record whose list is empty then produces no rows: decide whether that's what the report needs.

Mistakes to avoid: appending the inner list instead of its items (still nested), and losing the outer fields, so a row of {"tag": "csv"} no longer says which question it belongs to.

Review

Flattening builds new rows: an inner loop appends each inner value, not the inner list.
Lift nested values to clearly named keys; one row per inner item carries the outer fields down.
Nesting of unknown depth is a job for a recursive function.
Next: changing a value inside nested data, without changing someone else's copy.

Practise this