milestone 2/12 Integrate CRM + ERP with source authority
Combine CRM and ERP using clear source ownership rules.
CRM + ERP Integration · Follow extraction reconciliation, authoritative integration, date quarantine, quality gates, stable keys, replay-safe loads, and atomic promotion. Finish with a customer-grain capstone and field-level lineage.
approach
Normalize source identifiers under a declared contract: strip one leading NAS in ERP people and hyphens in ERP locations. Keep the latest CRM customer; source_row breaks equal timestamps. CRM gender wins when meaningful; ERP supplies a fallback. Preserve every valid CRM customer. Validate birth dates against fixed 2025-04-01.
WITH crm AS( WITH ranked AS ( SELECT *, ROW_NUMBER() OVER(PARTITION BY TRIM(customer_id) ORDER BY updated_at DESC,source_row DESC) AS rn FROM crm_people WHERE customer_id IS NOT NULL AND TRIM(customer_id)<>'' ) SELECT TRIM(customer_id) AS customer_id, TRIM(name) AS name, CASE UPPER(TRIM(gender)) WHEN 'F' THEN 'Female' WHEN 'M' THEN 'Male' ELSE NULL END AS crm_gender FROM ranked WHERE rn=1 ),erp AS( SELECT CASE WHEN SUBSTR(TRIM(erp_id),1,3)='NAS' THEN SUBSTR(TRIM(erp_id),4) ELSE TRIM(erp_id) END AS customer_id, CASE UPPER(TRIM(gender)) WHEN 'F' THEN 'Female' WHEN 'FEMALE' THEN 'Female' WHEN 'M' THEN 'Male' WHEN 'MALE' THEN 'Male' ELSE NULL END AS erp_gender, birth_date FROM erp_people ),loc AS( SELECT REPLACE(TRIM(erp_id),'-','') AS customer_id, CASE UPPER(TRIM(country_code)) WHEN 'US' THEN 'United States' WHEN 'USA' THEN 'United States' WHEN 'GB' THEN 'United Kingdom' WHEN 'UK' THEN 'United Kingdom' WHEN 'IND' THEN 'India' ELSE 'Unknown' END AS country FROM erp_locations ) SELECT c.customer_id, c.name, COALESCE(c.crm_gender,e.erp_gender,'Unknown') AS gender, COALESCE(l.country,'Unknown') AS country, CASE WHEN e.birth_date GLOB '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]' AND date(e.birth_date,'+0 days')=e.birth_date AND e.birth_date<='2025-04-01' THEN e.birth_date ELSE NULL END AS birth_date FROM crm c LEFT JOIN erp e ON e.customer_id=c.customer_id LEFT JOIN loc l ON l.customer_id=c.customer_id
data for this milestone
crm_people5 columns · 6 rows
| source_rowinteger · key | customer_idtext | nametext | gendertext | updated_attext |
|---|---|---|---|---|
| 1 | C001 | Alice | F | 2025-01-01 |
| 2 | C001 | Alice Nguyen | F | 2025-01-05 |
| 3 | C002 | Ben | n/a | 2025-01-02 |
| 4 | C003 | Cara | F | 2025-01-03 |
| 5 | C004 | Dev | NULL | 2025-01-04 |
showing 5 of 6 rows
erp_people3 columns · 5 rows
| erp_idtext | gendertext | birth_datetext |
|---|---|---|
| NASC001 | Male | 1990-05-12 |
| NASC002 | Male | 1988-02-29 |
| NASC003 | Female | 2028-01-01 |
| NASC004 | Other | 1995-06-20 |
| NASC999 | Female | 1980-01-01 |
all 5 rows
erp_locations2 columns · 4 rows
| erp_idtext | country_codetext |
|---|---|
| C-001 | USA |
| C-002 | GB |
| C-003 | UK |
| C-004 | IND |
all 4 rows