runtimeSeries and DataFrames
A DataFrame is a table with named columns, each column a Series of one dtype. It reads a CSV in one call and calculates whole columns at once.
- Build a DataFrame from a list of dictionaries and from CSV text with
pd.read_csv - Describe a table with
shape,columnsandhead, and take one column out as a Series - Add a column calculated from other columns, and explain why Series arithmetic lines values up by index label
Orders have an ID (text), a channel (text) and a total (a number). A NumPy array wants one dtype for everything, and a list of dictionaries needs a loop for every question. pandas gives you a DataFrame: a table whose columns each have a name and their own dtype. One column on its own is a Series: an array of values plus an index, the row labels (0, 1, 2, … unless you choose others).
pd.DataFrame(records) builds a table from a list of dictionaries, one per row. shape is (rows, columns), columns lists the names, and df["total"] takes one column out as a Series, which has sum(), mean(), max() and the rest.
Run each example to see what it prints. Running examples doesn't count toward progress.
import pandas as pd orders = pd.DataFrame([ {"order_id": "O100282", "channel": "web", "status": "delivered", "total": 35.84}, {"order_id": "O100376", "channel": "web", "status": "returned", "total": 91.47}, {"order_id": "O101723", "channel": "phone", "status": "delivered", "total": 54.55}, {"order_id": "O103199", "channel": "app", "status": "delivered", "total": 64.17}, ]) print(orders) print(orders.shape, list(orders.columns)) totals = orders["total"] print(type(totals).__name__, totals.max(), round(totals.sum(), 2))
import pandas as pd orders = pd.DataFrame([{"order_id": "O100282", "total": 35.84}]) print(orders["Total"].sum())
Most tables arrive as files. pd.read_csv(path) reads one, works out each column's dtype, and turns empty cells into missing values. Here the CSV text is wrapped in io.StringIO, which makes a string behave like an open file, so the example is self-contained; with a real file you pass its path. The reverse, df.to_csv(path, index=False), writes a table to a CSV file (the Data Formats & Analytical Workflows module covers export options).
Arithmetic between columns works value by value, like NumPy, and assigning to a new name adds a column.
import io import pandas as pd csv_text = """order_id,line_no,product_id,quantity,unit_price,discount_rate O100282,1,P0164,1,31,0.1 O100376,1,P0594,1,48,0 O100376,2,P0708,1,37.49,0 O100918,1,P0300,1,3.99,0 O100918,2,P0762,1,42.49,0 """ items = pd.read_csv(io.StringIO(csv_text)) items["line_total"] = items["quantity"] * items["unit_price"] * (1 - items["discount_rate"]) print(items.head(3)) print(len(items), "lines,", items["quantity"].sum(), "units")
Index alignment. When two Series are combined, pandas pairs values by index label, not by position. Columns of one DataFrame share an index, so that is invisible; but two separately built Series of different lengths give missing values (NaN) where a label exists in only one of them.
When to use a DataFrame: tabular data with named columns of different kinds, especially from files. Mistakes to avoid: misspelt column names; assigning a new column to a DataFrame your function was *given* (that changes the caller's table: build the result on a copy); and assuming position-based pairing between unrelated Series.
- A DataFrame is a table of named columns; each column is a Series with its own dtype and a shared index.
pd.DataFrame(records)andpd.read_csv(path)build tables;shape,columnsandhead()describe them.- Column arithmetic is element by element; add a column on a copy when the table was given to you.
- Next: pick rows and columns by label and by position.