lesson 77 Collections & Itertools · intermediate

Group without the setup line: defaultdict

A defaultdict builds the starting value for a key the first time the key is used, so grouping and totalling need no `if key not in …` line.

You'll be able to

Create a defaultdict with a factory such as list, int or Counter, passed without brackets
Group with groups[key].append(value) and total with totals[key] += amount
Know that reading a missing key creates it, and return dict(groups) to callers

The idea

Grouping in the Collection Algorithms module always had a setup line: if key not in groups: groups[key] = [], then the append. Forget it and the first new key raises KeyError.

defaultdict(list) removes that line. When you use a key it doesn't have yet, it calls list() to make an empty list, stores it under the key, and carries on with your append. The argument is the function that makes the starting value (a *factory*): list gives [], int gives 0, Counter gives an empty counter.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · Repositories each person pushed to

from collections import defaultdict

pushes = [
    ("maya-k", "quill-data/fast-parser"),
    ("devon-r", "tidewater/stream-sync"),
    ("maya-k", "tidewater/stream-sync"),
    ("lena-o", "quill-data/fast-parser"),
]
repos_by_user = defaultdict(list)
for login, repo in pushes:
    repos_by_user[login].append(repo)
print(repos_by_user["maya-k"])
print(dict(repos_by_user))

Pass the factory itself, without brackets. list() is a call that has already made one empty list, and a list can't make new values:

example-2.py · Calling the factory instead of passing it

from collections import defaultdict

repos_by_user = defaultdict(list())

The factory can be any tool that makes a starting value, including the Counter from the last lesson. A defaultdict(Counter) keeps one counter per key: a small two-level table.

example-3.py · Activity per repository, per event type

from collections import Counter, defaultdict

def activity_by_repo(events):
    activity = defaultdict(Counter)
    for event in events:
        activity[event["repo"]][event["type"]] += 1
    return activity

events = [
    {"repo": "quill-data/fast-parser", "type": "PushEvent"},
    {"repo": "tidewater/stream-sync", "type": "WatchEvent"},
    {"repo": "quill-data/fast-parser", "type": "PushEvent"},
    {"repo": "quill-data/fast-parser", "type": "IssuesEvent"},
]
activity = activity_by_repo(events)
print(activity["quill-data/fast-parser"])
print(activity["tidewater/stream-sync"]["PushEvent"], len(activity))

When to use it: grouping (defaultdict(list)), totals that aren't simple counts, such as minutes or money (defaultdict(int) or float), and nested tables. Mistakes to avoid: passing list() instead of list; and checking for a key by reading it: groups["x"] on a missing key creates "x" with an empty list, so test with in instead. A defaultdict also keeps creating keys after you return it, which surprises callers, so return dict(groups).

Review

defaultdict(factory) calls the factory (list, int, Counter…) the first time a key is used.
Pass the factory without brackets; group with .append, total with +=.
Reading a missing key creates it: test with in, and return dict(…) to callers.
Next: deque, a list built for taking items off the front, and for keeping only the newest few.

Practise this