lesson 93 Files & Directories · intermediate

Let with close the file, and read line by line

`with open(...) as f:` closes the file when the block ends, even when an error stops it. `for line in f` reads one line at a time, so a huge file never has to fit in memory.

You'll be able to

Use with open(...) as name: so the file is closed even if the block raises an error
Loop over a file object to get one line at a time, and remove the newline each line ends with
Read one file and write another in the same with statement

The idea

close() is easy to forget, and an error between open and close skips it entirely. An open file holds on to system resources, and text you wrote may not be saved until the file is closed.

with open(path, encoding="utf-8") as source: opens the file and names it source for the indented block. When the block ends, for any reason (the last line, a return, or an error), Python closes the file. From here on, every file in this course is opened this way.

A file object is also something you can loop over: for line in source: gives you one line per pass, including the \n at its end. Only one line is in memory at a time, so the same loop works on a three-line note and a five-gigabyte log.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · Each line, newline included

with open("readings.txt", "w", encoding="utf-8") as out:
    out.write("2024-01-01,6.7\n2024-01-02,7.2\n2024-01-03,3.3\n")

with open("readings.txt", encoding="utf-8") as source:
    for line in source:
        print(repr(line))
print(source.closed)

example-2.py · Writing after the block has closed the file

with open("readings.txt", "w", encoding="utf-8") as out:
    out.write("2024-01-01,6.7\n")
out.write("2024-01-02,7.2\n")

Processing line by line usually means: strip the newline, split the line into fields, convert, and keep only what the answer needs. This function finds the warmest day in a file of daily maximum temperatures (NOAA readings for one station) without ever holding the whole file.

example-3.py · The warmest day, one line at a time

def warmest_day(path):
    best_day = None
    best_temp = None
    with open(path, encoding="utf-8") as source:
        for line in source:
            day, temp_text = line.strip().split(",")
            temp = float(temp_text)
            if best_temp is None or temp > best_temp:
                best_day = day
                best_temp = temp
    return (best_day, best_temp)

with open("tmax.csv", "w", encoding="utf-8") as out:
    out.write("2024-01-01,6.7\n2024-01-02,7.2\n2024-01-03,3.3\n2024-01-04,7.2\n")
print(warmest_day("tmax.csv"))

When to use it: with for every file you open. for line in file whenever you can handle a file one line at a time (counting, filtering, finding, copying); read() only when you really need all the text at once. Mistakes to avoid: forgetting that each line ends with \n (print adds a second one, and the last field of a split keeps it), and looping over the same file object twice: the second loop starts at the end and finds nothing.

Review

with open(...) as name: closes the file when the block ends, even after an error or a return.
for line in file reads one line at a time; each line keeps its \n until you strip it.
A file object is used up after one pass; one with can open a source and a destination together.
Next: build paths and list folders with pathlib.

Practise this