lesson 210 pandas Foundations · advanced

Column data types

read_csv guesses each column's dtype from its contents. One bad value turns a price column into text; a leading zero vanishes from a code read as a number. Check dtypes, then convert on purpose.

You'll be able to

Read df.dtypes and recognise int64, float64, bool and text (str in pandas 3, object in pandas 2)
Convert with astype when every value is valid, and with pd.to_numeric(errors="coerce") when some aren't
Use read_csv(dtype={...}) to keep codes such as ZIP codes as text

The idea

read_csv looks at every value in a column and picks a dtype: int64 when all are whole numbers, float64 with decimals or gaps, bool for true/false, and text otherwise. In pandas 3 a text column shows as str; pandas 2 calls it object. A dtype decides what works: sum() on a text column joins strings instead of adding numbers.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · A product feed with one bad price

import io
import pandas as pd

csv_text = """sku,name,list_price,weight_kg
SPO-11681,Ultra Bike Light,7.49,2.75
HOM-22847,Deluxe Vase,31,0.3
ELE-24781,Organic Speaker,TBC,0.2
SPO-61044,Travel Yoga Mat,50.99,1.6
"""
products = pd.read_csv(io.StringIO(csv_text))
print(products.dtypes)
prices = pd.to_numeric(products["list_price"], errors="coerce")
print(prices.tolist(), prices.dtype)
print(products.loc[prices.isna(), "sku"].tolist())

example-2.py · astype on a column with bad text

import pandas as pd

prices = pd.Series(["7.49", "31", "n/a"])
print(prices.astype(float))

example-3.py · Codes that must stay text

import io
import pandas as pd

csv_text = """customer_id,zip,marketing_opt_in
C00038,02134,true
C00039,80202,false
C00065,07030,true
"""
guessed = pd.read_csv(io.StringIO(csv_text))
print(guessed["zip"].tolist(), guessed["zip"].dtype)
told = pd.read_csv(io.StringIO(csv_text), dtype={"zip": str})
print(told["zip"].tolist(), told["marketing_opt_in"].dtype)
print(told["marketing_opt_in"].sum(), "customers opted in")

When to use it: look at dtypes right after reading a file. Mistakes to avoid: computing with a column that is secretly text; letting IDs, ZIP codes or phone numbers become numbers; using astype where some values are expected to be bad (it raises), or errors="coerce" where a bad value should stop the program (it hides it, unless you count the NaNs it produced).

Review

read_csv guesses a dtype per column; check df.dtypes first (text is str in pandas 3, object in pandas 2).
astype converts when all values are valid; pd.to_numeric(errors="coerce") marks bad values as NaN.
read_csv(dtype={"zip": str}) keeps codes as text.
Next (the pandas Transformations module): group, join, pivot and clean whole tables.

Practise this