Project 04 · Warehouse and Reporting
Turn raw data into trusted reports.
Clean business records, build a tested model and create repeatable reports.
The brief
- The business question
- Profile raw CRM/sales records, build Silver and Gold tables in the disposable lab, explore the fact table, and publish repeatable reporting logic.
- What you’ll deliver
- Clean dimensions and facts, quality checks and customer, product and monthly reports.
- Skills
- Data cleaning · Data modeling · Reporting
- Scope
- Advanced level · 6 milestones · 6 tables
Available data
Fictional CRM/catalog/sales extracts and curated star-schema tables. Raw keys, dates, quantities, and sales need validation. One fact row is one sales line.
bronze_customers
raw_idtextnametextcountrytextupdated_attext
| # | raw_id | name | country | updated_at |
|---|---|---|---|---|
| 1 | C001 | Alice | USA | 2025-01-01 |
| 2 | C001 | Alice Nguyen | US | 2025-01-05 |
| 3 | C002 | Ben | GB | 2025-01-02 |
| 4 | C003 | Cara | UK | 2025-01-03 |
| 5 | C004 | Dev | IND | 2025-01-04 |
bronze_products
raw_idtextnametextcategorytextpricetext
| # | raw_id | name | category | price |
|---|---|---|---|---|
| 1 | P01 | Notebook | Stationery | 12 |
| 2 | P02 | Desk lamp | Home | 45 |
| 3 | P03 | Keyboard | Tech | 80 |
| 4 | P04 | Headphones | Tech | 120 |
bronze_sales
row_idintegerPKcustomer_idtextproduct_idtextquantitytextsalestextorder_datetext
| # | row_id | customer_id | product_id | quantity | sales | order_date |
|---|---|---|---|---|---|---|
| 1 | 1 | C001 | P01 | 2 | 24 | 2025-01-01 |
| 2 | 2 | C002 | P03 | 1 | 80 | 2025-01-02 |
| 3 | 3 | C001 | P02 | 1 | 45 | 2025-01-15 |
| 4 | 4 | C003 | P04 | 2 | 240 | 2025-02-01 |
| 5 | 5 | C004 | P03 | 2 | 160 | 2025-02-05 |
dim_customer
customer_keyintegerPKcustomer_idtextnametextcountrytext
| # | customer_key | customer_id | name | country |
|---|---|---|---|---|
| 1 | 1 | C001 | Alice Nguyen | United States |
| 2 | 2 | C002 | Ben | United Kingdom |
| 3 | 3 | C003 | Cara | United Kingdom |
| 4 | 4 | C004 | Dev | India |
| 5 | 5 | C005 | Elena | Canada |
dim_product
product_keyintegerPKproduct_idtextnametextcategorytextpricereal
| # | product_key | product_id | name | category | price |
|---|---|---|---|---|---|
| 1 | 1 | P01 | Notebook | Stationery | 12 |
| 2 | 2 | P02 | Desk lamp | Home | 45 |
| 3 | 3 | P03 | Keyboard | Tech | 80 |
| 4 | 4 | P04 | Headphones | Tech | 120 |
fact_sales
sale_keyintegerPKcustomer_keyintegerproduct_keyintegerquantityintegersalesrealorder_datetext
| # | sale_key | customer_key | product_key | quantity | sales | order_date |
|---|---|---|---|---|---|---|
| 1 | 1 | 1 | 1 | 2 | 24 | 2025-01-01 |
| 2 | 2 | 2 | 3 | 1 | 80 | 2025-01-02 |
| 3 | 3 | 1 | 2 | 1 | 45 | 2025-01-15 |
| 4 | 4 | 3 | 4 | 2 | 240 | 2025-02-01 |
| 5 | 5 | 4 | 3 | 2 | 160 | 2025-02-05 |
Milestones
StartBronze — inspect raw data
Silver — clean and deduplicate
Gold — model and test
Report
Project report
How a data warehouse is built
This project follows the standard layers: understand source data → load raw records (Bronze) → clean and validate (Silver) → build reporting tables (Gold) → produce reports.
Source contract
Structured fictional CRM, ERP and sales extracts. The foundation keeps latest state. Confirm source ownership, extraction counts, key formats, grain and refresh expectations before choosing architecture.
Bronze · preserve evidence
Raw source-shaped records retain spaces, sentinels, duplicates and malformed values. Compare extraction counts, inspect column alignment, and retain source row identity for replay and diagnosis.
Silver · standardize with explicit rules
Latest CRM row wins with a deterministic source_row tie-breaker. Normalize ERP keys under its format contract. Validate calendar dates, quarantine failures and reconcile every source row.
Gold · integrate business entities
CRM owns meaningful gender; ERP supplies fallback and country. Stable dimension keys link each validated sales line. Choose current-state or effective-time semantics before reporting. Views or stored tables are workload choices.
Reports · declare the metric
Customer totals use one row per customer and preserve zero-activity entities. Northstar sales means positive quantity × catalog price under a stated teaching rule, not an automatic accounting correction.