lesson 23 Joins & sets · intermediate
CROSS JOIN & a complete date grid
Build every combination of dates and entities.
the idea
CROSS JOIN pairs every row of one table with every row of another. Pairing stations with calendar dates builds a complete grid of expected station-days; a LEFT JOIN from that grid to the observations then shows which expected days have no data.
Dataset: Weather
The tables the challenges use, with their first 6 rows. Under each column name: the kind of value it holds, and which table it links to.
cities6 columns · 96 rows
| city_idnumber · unique id | city_nametext | state_codetext · links to states | latitudedecimal | longitudedecimal | land_area_sq_midecimal |
|---|---|---|---|---|---|
| 1 | Birmingham | AL | 33.527174 | -86.797036 | 147.07 |
| 2 | Mobile | AL | 30.668426 | -88.100226 | 139.482 |
| 3 | Anchorage | AK | 61.17425 | -149.284329 | 1707.004 |
| 4 | Fairbanks | AK | 64.836531 | -147.651745 | 31.746 |
| 5 | Phoenix | AZ | 33.572154 | -112.090132 | 518.331 |
| 6 | Tucson | AZ | 32.153036 | -110.870773 | 242.187 |
- city_idnumber · unique id
- 1
- city_nametext
- Birmingham
- state_codetext · links to states
- AL
- latitudedecimal
- 33.527174
- longitudedecimal
- -86.797036
- land_area_sq_midecimal
- 147.07
- city_idnumber · unique id
- 2
- city_nametext
- Mobile
- state_codetext · links to states
- AL
- latitudedecimal
- 30.668426
- longitudedecimal
- -88.100226
- land_area_sq_midecimal
- 139.482
- city_idnumber · unique id
- 3
- city_nametext
- Anchorage
- state_codetext · links to states
- AK
- latitudedecimal
- 61.17425
- longitudedecimal
- -149.284329
- land_area_sq_midecimal
- 1707.004
- city_idnumber · unique id
- 4
- city_nametext
- Fairbanks
- state_codetext · links to states
- AK
- latitudedecimal
- 64.836531
- longitudedecimal
- -147.651745
- land_area_sq_midecimal
- 31.746
- city_idnumber · unique id
- 5
- city_nametext
- Phoenix
- state_codetext · links to states
- AZ
- latitudedecimal
- 33.572154
- longitudedecimal
- -112.090132
- land_area_sq_midecimal
- 518.331
- city_idnumber · unique id
- 6
- city_nametext
- Tucson
- state_codetext · links to states
- AZ
- latitudedecimal
- 32.153036
- longitudedecimal
- -110.870773
- land_area_sq_midecimal
- 242.187
first 6 of 96 rows · your queries use all of them
stations10 columns · 100 rows
| station_idtext · unique id | station_nametext | city_idnumber · links to cities | is_primarynumber | latitudedecimal | longitudedecimal | elevation_mdecimal | first_observationdate | last_observationdate | reported_daysnumber |
|---|---|---|---|---|---|---|---|---|---|
| USW00003131 | San Diego Montgomery Fld | 10 | 1 | 32.8144 | -117.1375 | 127.4 | 2022-01-01 | 2024-12-31 | 1096 |
| USW00003184 | Phoenix Deer Valley Muni Airport | 5 | 1 | 33.6903 | -112.0661 | 453.2 | 2022-01-01 | 2024-12-31 | 1096 |
| USW00003822 | Savannah International Airport | 24 | 1 | 32.1314 | -81.2022 | 8.8 | 2022-01-01 | 2024-12-31 | 1096 |
| USW00003888 | Atlanta Fulton Co Airport | 23 | 1 | 33.7775 | -84.5247 | 245.1 | 2022-01-01 | 2024-12-31 | 1095 |
| USW00003928 | Wichita | 32 | 1 | 37.6544 | -97.4431 | 426.7 | 2022-01-01 | 2024-12-31 | 1096 |
| USW00003960 | Cahokia St Louis Dwtn Airport | 45 | 1 | 38.5639 | -90.1486 | 123.4 | 2022-01-01 | 2024-12-31 | 1096 |
- station_idtext · unique id
- USW00003131
- station_nametext
- San Diego Montgomery Fld
- city_idnumber · links to cities
- 10
- is_primarynumber
- 1
- latitudedecimal
- 32.8144
- longitudedecimal
- -117.1375
- elevation_mdecimal
- 127.4
- first_observationdate
- 2022-01-01
- last_observationdate
- 2024-12-31
- reported_daysnumber
- 1096
- station_idtext · unique id
- USW00003184
- station_nametext
- Phoenix Deer Valley Muni Airport
- city_idnumber · links to cities
- 5
- is_primarynumber
- 1
- latitudedecimal
- 33.6903
- longitudedecimal
- -112.0661
- elevation_mdecimal
- 453.2
- first_observationdate
- 2022-01-01
- last_observationdate
- 2024-12-31
- reported_daysnumber
- 1096
- station_idtext · unique id
- USW00003822
- station_nametext
- Savannah International Airport
- city_idnumber · links to cities
- 24
- is_primarynumber
- 1
- latitudedecimal
- 32.1314
- longitudedecimal
- -81.2022
- elevation_mdecimal
- 8.8
- first_observationdate
- 2022-01-01
- last_observationdate
- 2024-12-31
- reported_daysnumber
- 1096
- station_idtext · unique id
- USW00003888
- station_nametext
- Atlanta Fulton Co Airport
- city_idnumber · links to cities
- 23
- is_primarynumber
- 1
- latitudedecimal
- 33.7775
- longitudedecimal
- -84.5247
- elevation_mdecimal
- 245.1
- first_observationdate
- 2022-01-01
- last_observationdate
- 2024-12-31
- reported_daysnumber
- 1095
- station_idtext · unique id
- USW00003928
- station_nametext
- Wichita
- city_idnumber · links to cities
- 32
- is_primarynumber
- 1
- latitudedecimal
- 37.6544
- longitudedecimal
- -97.4431
- elevation_mdecimal
- 426.7
- first_observationdate
- 2022-01-01
- last_observationdate
- 2024-12-31
- reported_daysnumber
- 1096
- station_idtext · unique id
- USW00003960
- station_nametext
- Cahokia St Louis Dwtn Airport
- city_idnumber · links to cities
- 45
- is_primarynumber
- 1
- latitudedecimal
- 38.5639
- longitudedecimal
- -90.1486
- elevation_mdecimal
- 123.4
- first_observationdate
- 2022-01-01
- last_observationdate
- 2024-12-31
- reported_daysnumber
- 1096
first 6 of 100 rows · your queries use all of them
calendar10 columns · 1,096 rows
| datedate · unique id | yearnumber | monthnumber | day_of_yearnumber | day_of_weeknumber | meteorological_seasontext | day_of_monthnumber | day_nametext | is_weekendnumber | quarternumber |
|---|---|---|---|---|---|---|---|---|---|
| 2022-01-01 | 2022 | 1 | 1 | 6 | winter | 1 | Saturday | 1 | 1 |
| 2022-01-02 | 2022 | 1 | 2 | 0 | winter | 2 | Sunday | 1 | 1 |
| 2022-01-03 | 2022 | 1 | 3 | 1 | winter | 3 | Monday | 0 | 1 |
| 2022-01-04 | 2022 | 1 | 4 | 2 | winter | 4 | Tuesday | 0 | 1 |
| 2022-01-05 | 2022 | 1 | 5 | 3 | winter | 5 | Wednesday | 0 | 1 |
| 2022-01-06 | 2022 | 1 | 6 | 4 | winter | 6 | Thursday | 0 | 1 |
- datedate · unique id
- 2022-01-01
- yearnumber
- 2022
- monthnumber
- 1
- day_of_yearnumber
- 1
- day_of_weeknumber
- 6
- meteorological_seasontext
- winter
- day_of_monthnumber
- 1
- day_nametext
- Saturday
- is_weekendnumber
- 1
- quarternumber
- 1
- datedate · unique id
- 2022-01-02
- yearnumber
- 2022
- monthnumber
- 1
- day_of_yearnumber
- 2
- day_of_weeknumber
- 0
- meteorological_seasontext
- winter
- day_of_monthnumber
- 2
- day_nametext
- Sunday
- is_weekendnumber
- 1
- quarternumber
- 1
- datedate · unique id
- 2022-01-03
- yearnumber
- 2022
- monthnumber
- 1
- day_of_yearnumber
- 3
- day_of_weeknumber
- 1
- meteorological_seasontext
- winter
- day_of_monthnumber
- 3
- day_nametext
- Monday
- is_weekendnumber
- 0
- quarternumber
- 1
- datedate · unique id
- 2022-01-04
- yearnumber
- 2022
- monthnumber
- 1
- day_of_yearnumber
- 4
- day_of_weeknumber
- 2
- meteorological_seasontext
- winter
- day_of_monthnumber
- 4
- day_nametext
- Tuesday
- is_weekendnumber
- 0
- quarternumber
- 1
- datedate · unique id
- 2022-01-05
- yearnumber
- 2022
- monthnumber
- 1
- day_of_yearnumber
- 5
- day_of_weeknumber
- 3
- meteorological_seasontext
- winter
- day_of_monthnumber
- 5
- day_nametext
- Wednesday
- is_weekendnumber
- 0
- quarternumber
- 1
- datedate · unique id
- 2022-01-06
- yearnumber
- 2022
- monthnumber
- 1
- day_of_yearnumber
- 6
- day_of_weeknumber
- 4
- meteorological_seasontext
- winter
- day_of_monthnumber
- 6
- day_nametext
- Thursday
- is_weekendnumber
- 0
- quarternumber
- 1
first 6 of 1,096 rows · your queries use all of them
daily_weather10 columns · 109,214 rows
| station_idtext · links to stations | observation_datedate · links to calendar | tmax_cdecimal | tmin_cdecimal | precipitation_mmdecimal | precip_tracenumber | snowfall_mmdecimal | snow_depth_mmdecimal | avg_wind_speed_msdecimal | peak_gust_msdecimal |
|---|---|---|---|---|---|---|---|---|---|
| USW00003131 | 2022-01-01 | 17.2 | 7.2 | 0 | 0 | NULL | NULL | 2 | 6.3 |
| USW00003131 | 2022-01-02 | 20 | 5 | 0 | 0 | NULL | NULL | 1.1 | 5.4 |
| USW00003131 | 2022-01-03 | 17.8 | 3.9 | 0 | 0 | NULL | NULL | 1.1 | 4.5 |
| USW00003131 | 2022-01-04 | 17.2 | 4.4 | 0 | 0 | NULL | NULL | 1.4 | 4.5 |
| USW00003131 | 2022-01-05 | 21.1 | 5.6 | 0 | 0 | NULL | NULL | 0.9 | 4 |
| USW00003131 | 2022-01-06 | 22.2 | 7.2 | 0 | 0 | NULL | NULL | 0.6 | 3.6 |
- station_idtext · links to stations
- USW00003131
- observation_datedate · links to calendar
- 2022-01-01
- tmax_cdecimal
- 17.2
- tmin_cdecimal
- 7.2
- precipitation_mmdecimal
- 0
- precip_tracenumber
- 0
- snowfall_mmdecimal
- NULL
- snow_depth_mmdecimal
- NULL
- avg_wind_speed_msdecimal
- 2
- peak_gust_msdecimal
- 6.3
- station_idtext · links to stations
- USW00003131
- observation_datedate · links to calendar
- 2022-01-02
- tmax_cdecimal
- 20
- tmin_cdecimal
- 5
- precipitation_mmdecimal
- 0
- precip_tracenumber
- 0
- snowfall_mmdecimal
- NULL
- snow_depth_mmdecimal
- NULL
- avg_wind_speed_msdecimal
- 1.1
- peak_gust_msdecimal
- 5.4
- station_idtext · links to stations
- USW00003131
- observation_datedate · links to calendar
- 2022-01-03
- tmax_cdecimal
- 17.8
- tmin_cdecimal
- 3.9
- precipitation_mmdecimal
- 0
- precip_tracenumber
- 0
- snowfall_mmdecimal
- NULL
- snow_depth_mmdecimal
- NULL
- avg_wind_speed_msdecimal
- 1.1
- peak_gust_msdecimal
- 4.5
- station_idtext · links to stations
- USW00003131
- observation_datedate · links to calendar
- 2022-01-04
- tmax_cdecimal
- 17.2
- tmin_cdecimal
- 4.4
- precipitation_mmdecimal
- 0
- precip_tracenumber
- 0
- snowfall_mmdecimal
- NULL
- snow_depth_mmdecimal
- NULL
- avg_wind_speed_msdecimal
- 1.4
- peak_gust_msdecimal
- 4.5
- station_idtext · links to stations
- USW00003131
- observation_datedate · links to calendar
- 2022-01-05
- tmax_cdecimal
- 21.1
- tmin_cdecimal
- 5.6
- precipitation_mmdecimal
- 0
- precip_tracenumber
- 0
- snowfall_mmdecimal
- NULL
- snow_depth_mmdecimal
- NULL
- avg_wind_speed_msdecimal
- 0.9
- peak_gust_msdecimal
- 4
- station_idtext · links to stations
- USW00003131
- observation_datedate · links to calendar
- 2022-01-06
- tmax_cdecimal
- 22.2
- tmin_cdecimal
- 7.2
- precipitation_mmdecimal
- 0
- precip_tracenumber
- 0
- snowfall_mmdecimal
- NULL
- snow_depth_mmdecimal
- NULL
- avg_wind_speed_msdecimal
- 0.6
- peak_gust_msdecimal
- 3.6
first 6 of 109,214 rows · your queries use all of them
worked example
Run these steps in order to see the idea before you write your own query. The step marked “the pattern” is what the challenges ask for. Running the example does not count toward progress.
Pair every date with one station. Run this. CROSS JOIN builds every combination of station and calendar date in the filter.the pattern
example.sqlSELECT s.station_id, c.date FROM stations s CROSS JOIN calendar c WHERE s.station_id = 'USW00094728' AND c.date BETWEEN '2024-01-01' AND '2024-01-03';
Compare expected days to reported days. Run this. LEFT JOIN observations onto the grid; COUNT(d.observation_date) counts days with data.
step-2.sqlSELECT s.station_id, COUNT(*) AS expected_days, COUNT(d.observation_date) AS reported_days FROM stations s CROSS JOIN calendar c LEFT JOIN daily_weather d ON d.station_id = s.station_id AND d.observation_date = c.date GROUP BY s.station_id ORDER BY reported_days LIMIT 5;
common mistake
Grouping or averaging only the observations hides missing days entirely: a station that did not report looks the same as one that did.