lesson 176 Memory & Python Internals · advanced

References: names, aliases and when objects are freed

An assignment never copies anything: it attaches a name to an object. Two names, two list slots or a parameter can all refer to the same mutable object, and CPython frees that object only when the last reference to it is gone.

You'll be able to

Tell mutating an object (seen through every reference) from rebinding a name (affects only that name)
Spot shared objects created by * on a list of lists and by mutable default arguments
Explain reference counting: an object is freed when nothing refers to it any more

The idea

Python's variables are not boxes that hold values. Every value is an object somewhere in memory, and a name is a reference to it. saved = cart doesn't copy the cart; it gives the same list a second name. is asks whether two references point at the very same object, and id() gives the object's identity.

What matters is the difference between mutating the object (append, x[0] = …, and += on a list, which extends it in place) and rebinding a name (cart = cart + [...] builds a new list and moves only the name cart to it).

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · One cart, two names

cart = ["SPO-11681", "HOM-22847"]
saved = cart
saved.append("ELE-24781")
print(cart)
print(saved is cart, id(saved) == id(cart))
cart = cart + ["GRO-10442"]
print(saved)
print(saved is cart)

The same sharing hides inside containers. [row] * 3 makes a list holding three references to one row, not three rows. Function parameters work the same way: the parameter is one more reference to the caller's object, so a function that mutates its argument changes the caller's data.

example-2.py · Three rows that are one row

seats = [["."] * 4] * 3
seats[0][1] = "X"
for row in seats:
    print("".join(row))
print(seats[0] is seats[2])

A default argument is evaluated once, when def runs, so a mutable default is one object shared by every call that doesn't pass its own. This is the most famous reference bug in Python; the fix is None as the default and a new list inside the function.

example-3.py · The default list that remembers

def add_tag(tag, tags=[]):
    tags.append(tag)
    return tags

print(add_tag("fragile"))
print(add_tag("heavy"))

def add_tag_fixed(tag, tags=None):
    if tags is None:
        tags = []
    tags.append(tag)
    return tags

print(add_tag_fixed("fragile"))
print(add_tag_fixed("heavy"))

References also decide when memory is released. CPython keeps a reference count on every object: names, list slots, dictionary values and running frames each count once. When the count drops to zero the object is freed immediately. del name removes a name, not the object. The example uses __del__, a special method CPython calls just before freeing an object, to make the moment visible (other Python implementations, such as PyPy, free objects later, so never rely on __del__ for real cleanup).

example-4.py · Freed when the last reference goes

class Upload:
    def __init__(self, name):
        self.name = name

    def __del__(self):
        print("freed", self.name)

report = Upload("report.csv")
backup = report
del report
print("after del report")
backup = None
print("after backup = None")

When it matters: whenever a function receives a list or dictionary (mutate it only if that is the function's documented job, otherwise build a new one), when you build nested structures, and when a big object should be released (drop every reference to it). Mistakes to avoid: [[...]] * n for a grid, mutable default arguments, and assuming del frees an object that something else still refers to.

Review

Assignment, list slots and parameters create references, never copies.
Mutation shows through every reference; rebinding moves one name.
[[...]] * n and mutable default arguments share one object by accident.
CPython frees an object when its reference count reaches zero. Next: copying, shallow and deep.

Practise this