lesson 13 Calculations & data quality · intermediate
Filter groups with HAVING
Filter summaries after grouping rows.
the idea
WHERE filters input rows before grouping. HAVING filters groups after the aggregates are computed. Put a condition where its grain is: trip conditions in WHERE, group conditions in HAVING.
Dataset: Mobility
The tables the challenges use, with their first 6 rows. Under each column name: the kind of value it holds, and which table it links to.
trips19 columns · 128,620 rows
| trip_idnumber · unique id | vendor_idnumber · links to vendors | pickup_atdate & time | dropoff_atdate & time | passenger_countnumber | trip_distance_milesdecimal | rate_code_idnumber · links to rate_codes | store_and_fwd_flagtext | pickup_zone_idnumber · links to zones | dropoff_zone_idnumber · links to zones | payment_type_idnumber · links to payment_types | fare_amountdecimal | tip_amountdecimal | tolls_amountdecimal | surcharge_amountdecimal | congestion_surchargedecimal | airport_feedecimal | cbd_congestion_feedecimal | total_amountdecimal |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 2024-12-31 21:33:43 | 2024-12-31 21:39:00 | 2 | 1.12 | 1 | N | 179 | 7 | 2 | 7.9 | 0 | 0 | 2.5 | 0 | 0 | 0 | 10.4 |
| 2 | 2 | 2025-01-01 00:01:28 | 2025-01-01 00:21:12 | 1 | 5.51 | 1 | N | 249 | 112 | 1 | 25.4 | 7.47 | 6.94 | 2.5 | 2.5 | 0 | 0 | 44.81 |
| 3 | 2 | 2025-01-01 00:02:46 | 2025-01-01 00:21:27 | 1 | 2.98 | 1 | N | 158 | 162 | 1 | 18.4 | 5.85 | 0 | 2.5 | 2.5 | 0 | 0 | 29.25 |
| 4 | 2 | 2025-01-01 00:04:29 | 2025-01-01 00:55:58 | 9 | 31.97 | 5 | N | 132 | 265 | 2 | 90 | 0 | 20.32 | 1 | 0 | 0 | 0 | 111.32 |
| 5 | 2 | 2025-01-01 00:05:49 | 2025-01-01 00:20:00 | 2 | 3.45 | 1 | N | 263 | 107 | 1 | 17.7 | 4.54 | 0 | 2.5 | 2.5 | 0 | 0 | 27.24 |
| 6 | 2 | 2025-01-01 00:06:44 | 2025-01-01 00:32:07 | 1 | 8.04 | 1 | N | 132 | 191 | 2 | 35.9 | 0 | 0 | 2.5 | 0 | 0 | 0 | 38.4 |
- trip_idnumber · unique id
- 1
- vendor_idnumber · links to vendors
- 2
- pickup_atdate & time
- 2024-12-31 21:33:43
- dropoff_atdate & time
- 2024-12-31 21:39:00
- passenger_countnumber
- 2
- trip_distance_milesdecimal
- 1.12
- rate_code_idnumber · links to rate_codes
- 1
- store_and_fwd_flagtext
- N
- pickup_zone_idnumber · links to zones
- 179
- dropoff_zone_idnumber · links to zones
- 7
- payment_type_idnumber · links to payment_types
- 2
- fare_amountdecimal
- 7.9
- tip_amountdecimal
- 0
- tolls_amountdecimal
- 0
- surcharge_amountdecimal
- 2.5
- congestion_surchargedecimal
- 0
- airport_feedecimal
- 0
- cbd_congestion_feedecimal
- 0
- total_amountdecimal
- 10.4
- trip_idnumber · unique id
- 2
- vendor_idnumber · links to vendors
- 2
- pickup_atdate & time
- 2025-01-01 00:01:28
- dropoff_atdate & time
- 2025-01-01 00:21:12
- passenger_countnumber
- 1
- trip_distance_milesdecimal
- 5.51
- rate_code_idnumber · links to rate_codes
- 1
- store_and_fwd_flagtext
- N
- pickup_zone_idnumber · links to zones
- 249
- dropoff_zone_idnumber · links to zones
- 112
- payment_type_idnumber · links to payment_types
- 1
- fare_amountdecimal
- 25.4
- tip_amountdecimal
- 7.47
- tolls_amountdecimal
- 6.94
- surcharge_amountdecimal
- 2.5
- congestion_surchargedecimal
- 2.5
- airport_feedecimal
- 0
- cbd_congestion_feedecimal
- 0
- total_amountdecimal
- 44.81
- trip_idnumber · unique id
- 3
- vendor_idnumber · links to vendors
- 2
- pickup_atdate & time
- 2025-01-01 00:02:46
- dropoff_atdate & time
- 2025-01-01 00:21:27
- passenger_countnumber
- 1
- trip_distance_milesdecimal
- 2.98
- rate_code_idnumber · links to rate_codes
- 1
- store_and_fwd_flagtext
- N
- pickup_zone_idnumber · links to zones
- 158
- dropoff_zone_idnumber · links to zones
- 162
- payment_type_idnumber · links to payment_types
- 1
- fare_amountdecimal
- 18.4
- tip_amountdecimal
- 5.85
- tolls_amountdecimal
- 0
- surcharge_amountdecimal
- 2.5
- congestion_surchargedecimal
- 2.5
- airport_feedecimal
- 0
- cbd_congestion_feedecimal
- 0
- total_amountdecimal
- 29.25
- trip_idnumber · unique id
- 4
- vendor_idnumber · links to vendors
- 2
- pickup_atdate & time
- 2025-01-01 00:04:29
- dropoff_atdate & time
- 2025-01-01 00:55:58
- passenger_countnumber
- 9
- trip_distance_milesdecimal
- 31.97
- rate_code_idnumber · links to rate_codes
- 5
- store_and_fwd_flagtext
- N
- pickup_zone_idnumber · links to zones
- 132
- dropoff_zone_idnumber · links to zones
- 265
- payment_type_idnumber · links to payment_types
- 2
- fare_amountdecimal
- 90
- tip_amountdecimal
- 0
- tolls_amountdecimal
- 20.32
- surcharge_amountdecimal
- 1
- congestion_surchargedecimal
- 0
- airport_feedecimal
- 0
- cbd_congestion_feedecimal
- 0
- total_amountdecimal
- 111.32
- trip_idnumber · unique id
- 5
- vendor_idnumber · links to vendors
- 2
- pickup_atdate & time
- 2025-01-01 00:05:49
- dropoff_atdate & time
- 2025-01-01 00:20:00
- passenger_countnumber
- 2
- trip_distance_milesdecimal
- 3.45
- rate_code_idnumber · links to rate_codes
- 1
- store_and_fwd_flagtext
- N
- pickup_zone_idnumber · links to zones
- 263
- dropoff_zone_idnumber · links to zones
- 107
- payment_type_idnumber · links to payment_types
- 1
- fare_amountdecimal
- 17.7
- tip_amountdecimal
- 4.54
- tolls_amountdecimal
- 0
- surcharge_amountdecimal
- 2.5
- congestion_surchargedecimal
- 2.5
- airport_feedecimal
- 0
- cbd_congestion_feedecimal
- 0
- total_amountdecimal
- 27.24
- trip_idnumber · unique id
- 6
- vendor_idnumber · links to vendors
- 2
- pickup_atdate & time
- 2025-01-01 00:06:44
- dropoff_atdate & time
- 2025-01-01 00:32:07
- passenger_countnumber
- 1
- trip_distance_milesdecimal
- 8.04
- rate_code_idnumber · links to rate_codes
- 1
- store_and_fwd_flagtext
- N
- pickup_zone_idnumber · links to zones
- 132
- dropoff_zone_idnumber · links to zones
- 191
- payment_type_idnumber · links to payment_types
- 2
- fare_amountdecimal
- 35.9
- tip_amountdecimal
- 0
- tolls_amountdecimal
- 0
- surcharge_amountdecimal
- 2.5
- congestion_surchargedecimal
- 0
- airport_feedecimal
- 0
- cbd_congestion_feedecimal
- 0
- total_amountdecimal
- 38.4
first 6 of 128,620 rows · your queries use all of them
worked example
Run these steps in order to see the idea before you write your own query. The step marked “the pattern” is what the challenges ask for. Running the example does not count toward progress.
Count trips in each payment type. Run this. GROUP BY gives one row per payment_type_id with a trip count.
step-1.sqlSELECT payment_type_id, COUNT(*) AS trips FROM trips GROUP BY payment_type_id;
Drop small groups with HAVING. Run this. HAVING runs after grouping and keeps only groups with more than 10,000 trips.the pattern
example.sqlSELECT payment_type_id, COUNT(*) AS trips FROM trips GROUP BY payment_type_id HAVING COUNT(*) > 10000;
common mistake
WHERE fare_amount > 20 removes cheap trips; HAVING AVG(fare_amount) > 20 removes zones whose average is low. They answer different questions.