lesson 231 Secure Python · advanced

Accept only the input you expect

Anything that comes from outside (a form, a URL, a file, an API) may have been written by someone trying to break your program. Decide exactly what a valid value looks like and reject everything else, instead of trying to clean up what's dangerous.

You'll be able to

Explain why an allow-list (accept only the known good) is safer than a deny-list (block the known bad)
Check text with re.fullmatch and [0-9], not re.search or \d, and convert numbers inside try
Validate a whole request: known fields only, each with its rule, every problem reported

The idea

Every value that crosses into your program from outside is untrusted: a query string, a form field, a row of an uploaded CSV. Most of it is honest; some of it is mistyped; a little of it is built to make your program do something you never intended.

There are two ways to defend. A deny-list blocks values known to be dangerous: no <script>, no ;. It always misses something (<SCRIPT>, a look-alike character, a trick nobody thought of yet). An allow-list describes what a good value *is* (an order ID is O followed by six digits; a sort order is one of three words; a quantity is a whole number from 1 to 99) and rejects everything else. It is shorter, easier to test, and doesn't depend on guessing the attacker's ideas.

Two details matter with patterns. re.search finds a match *anywhere*, so O100282; DROP TABLE orders contains a valid ID and passes; re.fullmatch requires the whole text to fit. And \d matches any Unicode digit (Arabic-Indic ١٢ too), so write [0-9] when you mean the ten ASCII digits.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · search finds a match; fullmatch checks the whole value

import re

ORDER_ID = r"O[0-9]{6}"
for text in ["O100282", "O100282; DROP TABLE orders", "o100282", " O100282", "O١٠٠٢٨٢"]:
    found = re.search(ORDER_ID, text) is not None
    whole = re.fullmatch(ORDER_ID, text) is not None
    print(repr(text), found, whole)

Numbers need the same care. int() is generous: it accepts spaces around the digits, a + sign and other scripts' digits, and it raises ValueError for anything it can't read. Convert inside try, then check the range: a quantity of -3 or 100000 is a valid integer and still not a valid quantity.

example-2.py · Trusting a form field to be a whole number

form = {"sku": "SPO-11681", "quantity": "2.5"}
quantity = int(form["quantity"])
print("adding", quantity)

A real request has several fields. Validate all of them and report every problem, like a schema check (see Reliable Data Pipelines), but with the stricter attitude of security: unknown fields are rejected too, because a field you don't expect ("is_admin": "true") is exactly how some attacks start.

example-3.py · Validate an update-order request

import re

STATUSES = {"processing", "shipped", "cancelled"}
SKU = r"[A-Z]{3}-[0-9]{5}"

def check_update(request):
    errors = []
    for field in request:
        if field not in {"sku", "quantity", "status"}:
            errors.append(f"unknown field {field}")
    if not isinstance(request.get("sku"), str) or not re.fullmatch(SKU, request["sku"]):
        errors.append("sku must look like SPO-11681")
    try:
        quantity = int(request.get("quantity", ""))
    except ValueError:
        errors.append("quantity must be a whole number")
    else:
        if not 1 <= quantity <= 99:
            errors.append("quantity must be 1 to 99")
    if request.get("status") not in STATUSES:
        errors.append("status must be one of " + ", ".join(sorted(STATUSES)))
    return errors

print(check_update({"sku": "SPO-11681", "quantity": "2", "status": "shipped"}))
print(check_update({"sku": "spo-11681", "quantity": "0", "status": "shipped"}))
print(check_update({"sku": "HOM-22847", "quantity": "1", "status": "refunded", "is_admin": "true"}))

When to use it: at every entry point: web handlers, command-line arguments (see CLI & Automation), uploaded files, messages from other services. Mistakes to avoid: re.search or re.match where you mean the whole value; \d for ASCII digits; deny-lists of bad characters; silently "fixing" input (stripping ; and carrying on) instead of rejecting it; and checking the type but not the range.

Review

Untrusted input is accepted only when it matches what a good value is: a full pattern, a set of choices, a range.
re.fullmatch and [0-9], not re.search/re.match and \d; convert numbers inside try and check the range.
Reject unknown fields and bad values with reasons; never "clean" dangerous input and carry on.
Next: some values must never appear anywhere, not even in a log. Handling secrets.

Practise this