lesson 78 Warehouse & reporting projects · advanced
Integrate CRM + ERP with source authority
Combine CRM and ERP using clear source ownership rules.
the idea
Integration follows a written contract. Here: keep the latest CRM version per trimmed key (source_row breaks equal timestamps); normalize ERP keys by removing a leading NAS and location keys by removing hyphens; CRM gender wins when it is F or M, ERP gender is the fallback; countries come from the ISO code table; birth dates must be valid and not after 2011-12-31. Every keyed CRM customer is kept.
Dataset: Retail
The tables the challenges use, with their first 6 rows. Under each column name: the kind of value it holds, and which table it links to.
country_codes2 columns · 26 rows
| codetext · unique id | countrytext |
|---|---|
| AE | United Arab Emirates |
| AT | Austria |
| AU | Australia |
| BE | Belgium |
| BH | Bahrain |
| BR | Brazil |
- codetext · unique id
- AE
- countrytext
- United Arab Emirates
- codetext · unique id
- AT
- countrytext
- Austria
- codetext · unique id
- AU
- countrytext
- Australia
- codetext · unique id
- BE
- countrytext
- Belgium
- codetext · unique id
- BH
- countrytext
- Bahrain
- codetext · unique id
- BR
- countrytext
- Brazil
first 6 of 26 rows · your queries use all of them
crm_people5 columns · 1,707 rows
| source_rownumber · unique id | customer_idtext | contact_nametext | gendertext | updated_atdate & time |
|---|---|---|---|---|
| 1 | 12347 | Arthur Turner | n/a | 2010-10-31 10:00:00 |
| 2 | 12347 | Arthur Turner | n/a | 2010-11-01 10:00:00 |
| 3 | 12353 | Harry Turner | M | 2010-10-27 10:00:00 |
| 4 | 12359 | Theo Turner | M | 2009-12-05 10:00:00 |
| 5 | 12359 | Theo Turner | M | 2010-01-01 10:00:00 |
| 6 | 12359 | Theo Turner | M | 2011-11-01 10:00:00 |
- source_rownumber · unique id
- 1
- customer_idtext
- 12347
- contact_nametext
- Arthur Turner
- gendertext
- n/a
- updated_atdate & time
- 2010-10-31 10:00:00
- source_rownumber · unique id
- 2
- customer_idtext
- 12347
- contact_nametext
- Arthur Turner
- gendertext
- n/a
- updated_atdate & time
- 2010-11-01 10:00:00
- source_rownumber · unique id
- 3
- customer_idtext
- 12353
- contact_nametext
- Harry Turner
- gendertext
- M
- updated_atdate & time
- 2010-10-27 10:00:00
- source_rownumber · unique id
- 4
- customer_idtext
- 12359
- contact_nametext
- Theo Turner
- gendertext
- M
- updated_atdate & time
- 2009-12-05 10:00:00
- source_rownumber · unique id
- 5
- customer_idtext
- 12359
- contact_nametext
- Theo Turner
- gendertext
- M
- updated_atdate & time
- 2010-01-01 10:00:00
- source_rownumber · unique id
- 6
- customer_idtext
- 12359
- contact_nametext
- Theo Turner
- gendertext
- M
- updated_atdate & time
- 2011-11-01 10:00:00
first 6 of 1,707 rows · your queries use all of them
erp_people3 columns · 848 rows
| erp_idtext · unique id | gendertext | birth_datedate |
|---|---|---|
| 12389 | female | 1978-06-14 |
| 12392 | Female | 1981-09-17 |
| 12416 | Male | 2031-05-01 |
| 12373 | Male | 1962-02-26 |
| 12374 | Male | 1963-03-27 |
| 12817 | female | 1983-02-22 |
- erp_idtext · unique id
- 12389
- gendertext
- female
- birth_datedate
- 1978-06-14
- erp_idtext · unique id
- 12392
- gendertext
- Female
- birth_datedate
- 1981-09-17
- erp_idtext · unique id
- 12416
- gendertext
- Male
- birth_datedate
- 2031-05-01
- erp_idtext · unique id
- 12373
- gendertext
- Male
- birth_datedate
- 1962-02-26
- erp_idtext · unique id
- 12374
- gendertext
- Male
- birth_datedate
- 1963-03-27
- erp_idtext · unique id
- 12817
- gendertext
- female
- birth_datedate
- 1983-02-22
first 6 of 848 rows · your queries use all of them
erp_locations2 columns · 848 rows
| erp_idtext · unique id | country_codetext |
|---|---|
| 12-389 | AU |
| 12-392 | AU |
| 12-416 | AU |
| 12-373 | AT |
| 12-374 | AT |
| 12-817 | AT |
- erp_idtext · unique id
- 12-389
- country_codetext
- AU
- erp_idtext · unique id
- 12-392
- country_codetext
- AU
- erp_idtext · unique id
- 12-416
- country_codetext
- AU
- erp_idtext · unique id
- 12-373
- country_codetext
- AT
- erp_idtext · unique id
- 12-374
- country_codetext
- AT
- erp_idtext · unique id
- 12-817
- country_codetext
- AT
first 6 of 848 rows · your queries use all of them
worked example
Run these steps in order to see the idea before you write your own query. The step marked “the pattern” is what the challenges ask for. Running the example does not count toward progress.
Strip NAS from ERP ids. Some erp_id values start with NAS; the contract removes that prefix so ERP keys align with CRM customer_id.the pattern
example.sqlSELECT erp_id, CASE WHEN substr(TRIM(erp_id), 1, 3) = 'NAS' THEN substr(TRIM(erp_id), 4) ELSE TRIM(erp_id) END AS customer_id FROM erp_people LIMIT 10;
Remove hyphens from location ids. erp_locations uses hyphenated ids; REPLACE lets you join locations to the same normalized customer_id as people.
step-2.sqlSELECT erp_id, REPLACE(erp_id, '-', '') AS customer_id, country_code FROM erp_locations LIMIT 10;
Group ERP gender spellings. ERP uses FEMALE and MALE while CRM uses F and M—you need explicit mapping before COALESCE can pick a fallback.
step-3.sqlSELECT UPPER(TRIM(gender)) AS erp_gender, COUNT(*) AS customers FROM erp_people GROUP BY UPPER(TRIM(gender));
common mistake
COALESCE(crm_gender, erp_gender) only works if sentinel values like 'n/a' and '' are first turned into NULL.