lesson 105 Dates Numbers & Regex · intermediate

Patterns in text with regular expressions

A regular expression describes a shape of text, such as "O followed by six digits". `re.search` and `re.findall` find it, groups pull out its parts, `re.sub` rewrites it and `re.fullmatch` checks that a whole value has it.

You'll be able to

Write patterns with \d, [A-Z], {n}, +, \s and \b, in raw strings
Pull out parts of a match with groups, and find every match with findall
Rewrite matches with re.sub, and validate a whole value with re.fullmatch

The idea

String methods find fixed text: "O104839" in message. Real questions are about shapes: any order ID (an O and six digits), any SKU (three capitals, a dash, five digits), any run of spaces. A regular expression describes such a shape.

The pieces you'll use most: \d a digit, [A-Z] one capital letter, {6} exactly six of the previous thing, + one or more, \s a space or tab, \b a word boundary, and parentheses ( ) for a group you want to pull out. Write patterns as raw strings, r"O\d{6}", so Python leaves the backslashes for re.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · Order IDs in a support message

import re

message = "Hi, order O104839 arrived damaged; O104900 is fine. Refund to card ending 4242."
match = re.search(r"\bO(\d{6})\b", message)
print(match.group(0), match.group(1))
print(re.findall(r"O\d{6}", message))
print(re.findall(r"\d{4}", message))
print(re.findall(r"\b\d{4}\b", message))

example-2.py · Using a match that didn't happen

import re

match = re.search(r"O\d{6}", "Where is my parcel?")
print(match.group(0))

re.sub(pattern, replacement, text) replaces every match, and re.fullmatch(pattern, text) matches only if the whole text has the shape, which is what validation needs (re.search would accept a valid code buried in junk). These two clean the double spaces out of Online Retail descriptions and validate Runtime Market SKUs.

example-3.py · Tidy descriptions and validate SKUs

import re

def tidy(description):
    return re.sub(r"\s+", " ", description).strip()

def valid_sku(code):
    return re.fullmatch(r"[A-Z]{3}-\d{5}", code) is not None

print(tidy("JUMBO  BAG BAROQUE   BLACK WHITE "))
for code in ["HOM-52782", "hom-52782", "HOM-527821", "SKU HOM-52782"]:
    print(code, valid_sku(code))

When to use it: extracting IDs, dates and amounts from free text, cleaning inconsistent spacing, and checking that a value has the right shape. Mistakes to avoid: patterns that are too loose (no \b or fullmatch), forgetting that search can return None, and using a regex where a string method or a real parser (csv, json, datetime) would be clearer.

Review

Patterns describe shapes: \d, [A-Z], {n}, +, \s, \b; write them as raw strings.
re.search finds the first match (or None), re.findall every match; groups pull out parts.
re.sub rewrites matches (with \1, \2 for groups); re.fullmatch validates a whole value.
Next module: splitting your own code into modules and packages.

Practise this