lesson 234 Secure Python · advanced

Load data without running it

Reading a saved file or a request body should only ever produce data. A loader that lets the data choose what to call or which object to build hands control to whoever wrote the data. Plain JSON can only describe data; read untrusted input with it, allow-list any types it may create, and check its shape.

You'll be able to

Show how a loader that looks up names from the data can be made to call a function nobody intended
Build objects from JSON through an allow-list of types, and validate the shape before using it
Write and read values JSON doesn't support (Decimal, datetime) by converting them explicitly

The idea

Serialization turns values into text or bytes to store or send, and loading turns them back. The danger is in loading. Some loaders let the data say *what to build or call*: Python's built-in eval runs text as code, serialisers that rebuild arbitrary Python objects store which function to call for each object, and home-made JSON decoders often look up a class by a name found in the data. With any of them, whoever wrote the data (an uploaded file, a cookie, a message from another service) chooses what your program runs.

Plain JSON can only describe dictionaries, lists, strings, numbers, booleans and null, so json.loads on hostile text gives you plain data or raises JSONDecodeError. When you *do* want objects back, choose the classes from an allow-list you wrote, never by looking up whatever name the data contains. And loaded data still needs a shape check (the Reliable Data Pipelines module, L01): valid JSON is not the same as a valid cart.

The example builds the classic mistake for real: a decoder hook (object_hook, which json.loads calls for every JSON object it reads) that creates objects by looking up the "type" named in the data.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · A decoder that lets the data choose what to call

import json

class Refund:
    def __init__(self, order_id, amount):
        self.order_id = order_id
        self.amount = amount

    def __repr__(self):
        return f"Refund({self.order_id}, {self.amount})"

def clear_payment_logs():
    print("payment logs deleted!")

def build_unsafe(data):
    if "type" in data:
        factory = globals()[data.pop("type")]
        return factory(**data)
    return data

ALLOWED = {"Refund": Refund}

def build_safe(data):
    if "type" in data:
        kind = data.pop("type")
        if kind not in ALLOWED:
            raise ValueError(f"cannot build {kind!r}")
        return ALLOWED[kind](**data)
    return data

honest = '{"type": "Refund", "order_id": "O100376", "amount": 91.47}'
hostile = '{"type": "clear_payment_logs"}'

print(json.loads(honest, object_hook=build_unsafe))
json.loads(hostile, object_hook=build_unsafe)
print(json.loads(honest, object_hook=build_safe))
try:
    json.loads(hostile, object_hook=build_safe)
except ValueError as err:
    print(err)

The price of a data-only format is that it holds only data. JSON has no Decimal, datetime or set, and json.dumps refuses them rather than guess:

example-2.py · JSON doesn't guess how to store a Decimal

import json
from decimal import Decimal

order = {"order_id": "O100282", "total": Decimal("35.84")}
print(json.dumps(order))

Loaded JSON is still untrusted. A saved cart might be a list instead of an object, have a quantity of -5 or "2", or carry extra fields. Check the shape before anything uses it, and turn every problem into one kind of error your caller can handle.

example-3.py · Load a saved cart and check its shape

import json
import re

def load_cart(text):
    try:
        data = json.loads(text)
    except json.JSONDecodeError as err:
        raise ValueError(f"cart is not valid JSON: {err.msg}")
    if not isinstance(data, dict) or not isinstance(data.get("items"), list):
        raise ValueError("cart must be an object with an items list")
    lines = []
    for item in data["items"]:
        if not isinstance(item, dict):
            raise ValueError("each item must be an object")
        sku, qty = item.get("sku"), item.get("qty")
        if not isinstance(sku, str) or not re.fullmatch(r"[A-Z]{3}-[0-9]{5}", sku):
            raise ValueError(f"bad sku {sku!r}")
        if isinstance(qty, bool) or not isinstance(qty, int) or not 1 <= qty <= 99:
            raise ValueError(f"bad qty {qty!r} for {sku}")
        lines.append((sku, qty))
    return lines

print(load_cart('{"items": [{"sku": "SPO-11681", "qty": 2}, {"sku": "HOM-22847", "qty": 1}]}'))
for text in ['{"items": [{"sku": "SPO-11681", "qty": "2"}]}', '["SPO-11681"]', "{'items': []}"]:
    try:
        load_cart(text)
    except ValueError as err:
        print(err)

When to use it: for anything that arrives from outside or is stored where others could change it: request bodies, uploads, cookies, cache files, messages. Formats that rebuild arbitrary objects are only for data your own program wrote and nobody else can touch. Mistakes to avoid: evaluating text to "parse" it; choosing a class or function by a name taken from the data; assuming loaded JSON has the shape you saved; and converting Decimal money to float to make JSON accept it (str keeps the exact value).

Review

Loading must never run code: no evaluating outside text, no object-rebuilding loaders on untrusted bytes, no names from the data looked up in globals().
Read untrusted data with json.loads; create objects only from an allow-list of types; validate the shape.
Convert values JSON lacks yourself: Decimal → str, datetime → isoformat(), and back.
Next: a file name from a user can point outside the folder you meant. Path security.

Practise this

Make a record JSON-safe