milestone 12/12 Capstone: integrated customers to trusted sales
Combine integration and quality checks into trusted sales reporting.
CRM + ERP Integration · Follow extraction reconciliation, authoritative integration, date quarantine, quality gates, stable keys, replay-safe loads, and atomic promotion. Finish with a customer-grain capstone and field-level lineage.
approach
Build two reusable temporary outputs in one script: integrated CRM/ERP customers and validated sales lines with existing dimension keys. Report at customer grain, preserving customers with no valid sales. The catalog states field authority and exercise sales rules. This is a runnable miniature of the architecture, not a deployed enterprise pipeline.
CREATE TEMP TABLE integrated_customers AS
WITH crm AS(
WITH ranked AS (
SELECT
*,
ROW_NUMBER() OVER(PARTITION BY TRIM(customer_id) ORDER BY updated_at DESC,source_row DESC) AS rn
FROM crm_people
WHERE customer_id IS NOT NULL AND TRIM(customer_id)<>''
)
SELECT
TRIM(customer_id) AS customer_id,
TRIM(name) AS name,
CASE UPPER(TRIM(gender))
WHEN 'F' THEN 'Female'
WHEN 'M' THEN 'Male'
ELSE NULL
END AS crm_gender
FROM ranked
WHERE rn=1
),erp AS(
SELECT
CASE
WHEN SUBSTR(TRIM(erp_id),1,3)='NAS' THEN SUBSTR(TRIM(erp_id),4)
ELSE TRIM(erp_id)
END AS customer_id,
CASE UPPER(TRIM(gender))
WHEN 'F' THEN 'Female'
WHEN 'FEMALE' THEN 'Female'
WHEN 'M' THEN 'Male'
WHEN 'MALE' THEN 'Male'
ELSE NULL
END AS erp_gender,
birth_date
FROM erp_people
),loc AS(
SELECT
REPLACE(TRIM(erp_id),'-','') AS customer_id,
CASE UPPER(TRIM(country_code))
WHEN 'US' THEN 'United States'
WHEN 'USA' THEN 'United States'
WHEN 'GB' THEN 'United Kingdom'
WHEN 'UK' THEN 'United Kingdom'
WHEN 'IND' THEN 'India'
ELSE 'Unknown'
END AS country
FROM erp_locations
)
SELECT
c.customer_id,
c.name,
COALESCE(c.crm_gender,e.erp_gender,'Unknown') AS gender,
COALESCE(l.country,'Unknown') AS country,
CASE
WHEN e.birth_date GLOB '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
AND date(e.birth_date,'+0 days')=e.birth_date
AND e.birth_date<='2025-04-01'
THEN e.birth_date
ELSE NULL
END AS birth_date
FROM crm c
LEFT JOIN erp e ON e.customer_id=c.customer_id
LEFT JOIN loc l ON l.customer_id=c.customer_id;
CREATE TEMP TABLE built_sales AS
SELECT
b.row_id AS sale_key,
c.customer_key,
p.product_key,
CAST(b.quantity AS INTEGER) AS quantity,
CAST(b.quantity AS INTEGER)*p.price AS sales,
b.order_date
FROM bronze_sales b
JOIN dim_customer c ON c.customer_id=TRIM(b.customer_id)
JOIN dim_product p ON p.product_id=TRIM(b.product_id)
WHERE TRIM(b.quantity)<>''
AND TRIM(b.quantity) NOT GLOB '*[^0-9]*'
AND CAST(b.quantity AS INTEGER)>0
AND b.order_date GLOB '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
AND date(b.order_date,'+0 days')=b.order_date;
SELECT
i.customer_id,
i.name,
i.gender,
i.country,
COUNT(s.sale_key) AS sales_lines,
COALESCE(SUM(s.sales),0) AS total_sales
FROM integrated_customers i
JOIN dim_customer c ON c.customer_id=i.customer_id
LEFT JOIN built_sales s ON s.customer_key=c.customer_key
GROUP BY i.customer_id,i.name,i.gender,i.country;data for this milestone
crm_people5 columns · 6 rows
| source_rowinteger · key | customer_idtext | nametext | gendertext | updated_attext |
|---|---|---|---|---|
| 1 | C001 | Alice | F | 2025-01-01 |
| 2 | C001 | Alice Nguyen | F | 2025-01-05 |
| 3 | C002 | Ben | n/a | 2025-01-02 |
| 4 | C003 | Cara | F | 2025-01-03 |
| 5 | C004 | Dev | NULL | 2025-01-04 |
showing 5 of 6 rows
erp_people3 columns · 5 rows
| erp_idtext | gendertext | birth_datetext |
|---|---|---|
| NASC001 | Male | 1990-05-12 |
| NASC002 | Male | 1988-02-29 |
| NASC003 | Female | 2028-01-01 |
| NASC004 | Other | 1995-06-20 |
| NASC999 | Female | 1980-01-01 |
all 5 rows
erp_locations2 columns · 4 rows
| erp_idtext | country_codetext |
|---|---|
| C-001 | USA |
| C-002 | GB |
| C-003 | UK |
| C-004 | IND |
all 4 rows
bronze_sales6 columns · 13 rows
| row_idinteger · key | customer_idtext | product_idtext | quantitytext | salestext | order_datetext |
|---|---|---|---|---|---|
| 1 | C001 | P01 | 2 | 24 | 2025-01-01 |
| 2 | C002 | P03 | 1 | 80 | 2025-01-02 |
| 3 | C001 | P02 | 1 | 45 | 2025-01-15 |
| 4 | C003 | P04 | 2 | 240 | 2025-02-01 |
| 5 | C004 | P03 | 2 | 160 | 2025-02-05 |
showing 5 of 13 rows
dim_customer4 columns · 5 rows
| customer_keyinteger · key | customer_idtext | nametext | countrytext |
|---|---|---|---|
| 1 | C001 | Alice Nguyen | United States |
| 2 | C002 | Ben | United Kingdom |
| 3 | C003 | Cara | United Kingdom |
| 4 | C004 | Dev | India |
| 5 | C005 | Elena | Canada |
all 5 rows
dim_product5 columns · 4 rows
| product_keyinteger · key | product_idtext | nametext | categorytext | pricereal |
|---|---|---|---|---|
| 1 | P01 | Notebook | Stationery | 12 |
| 2 | P02 | Desk lamp | Home | 45 |
| 3 | P03 | Keyboard | Tech | 80 |
| 4 | P04 | Headphones | Tech | 120 |
all 4 rows