lesson 48 Fundamentals Integration · beginner

Analyse a run of readings

Measurements come in order, with gaps. Analysing them means skipping missing values honestly, comparing each reading with its neighbour, and looking at windows of days.

You'll be able to

Skip None readings with is None and report how many were missing
Compare each reading with the one before it using positions
Compute a value over a window of consecutive readings with a slice

The idea

NOAA's daily records come from real stations, and real stations miss days. In the data those days are empty, which in Python becomes None. An honest summary skips the gaps and says how many there were: an average of 20 readings out of 31 days is not the same claim as an average of 31.

None can't be compared with a number (None > 5 is a TypeError), so test is None before doing anything else with a reading.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · A month's highs, with gaps

def summary(highs):
    """Return (days with data, missing days, average high) for daily highs that may be None."""
    total = 0
    count = 0
    missing = 0
    for high in highs:
        if high is None:
            missing = missing + 1
            continue
        total = total + high
        count = count + 1
    average = total / count if count > 0 else None
    return count, missing, average

highs = [31.1, 33.9, None, 35.0, 29.4, None, 32.2]
count, missing, average = summary(highs)
print(f"{count} days, {missing} missing, average {average:.1f}")
print(summary([None, None]))

Many weather questions compare a day with the day before: *the first day it warmed past 20 °C*, *how often it got colder overnight*. A for loop over values can't see the previous value, so loop over positions with range(1, len(temps)) and read temps[i - 1] and temps[i]. Starting at 1 means the first day has a day before it.

Other questions look at a window of days: a slice temps[i:i + 3] is the three days starting at i.

example-2.py · First warm day and the warmest three days

def first_crossing(temps, level):
    """Return the position of the first day that rose to level from below it, or None."""
    for i in range(1, len(temps)):
        if temps[i - 1] < level <= temps[i]:
            return i
    return None

def warmest_window(temps, days=3):
    """Return the start position of the warmest run of that many readings."""
    best_start = 0
    best_total = None
    for start in range(len(temps) - days + 1):
        total = 0
        for t in temps[start:start + days]:
            total = total + t
        if best_total is None or total > best_total:
            best_start = start
            best_total = total
    return best_start

temps = [18.3, 21.7, 14.2, 15.0, 22.8, 24.1, 23.5]
print(first_crossing(temps, 20), first_crossing(temps, 22), first_crossing(temps, 30))
print(warmest_window(temps), warmest_window(temps, 2))

When to use it: for anything measured over time: temperatures, prices, step counts, server response times. Mistakes to avoid:

Comparing or adding None: test is None first.
Starting a neighbour loop at 0: temps[-1] is the last reading, so day 0 is silently compared with the end of the list.
Dividing by the number of days instead of the number of readings.

Review

Skip None readings with is None before comparing, and say how many were missing.
Compare neighbours by position, starting at 1; look at windows with a slice.
Divide by the number of readings you actually used.
Next: sort free-text messages into categories by the words they contain.

Practise this

Weekly totals from daily steps