Sep 2023 – Jan 2024
CAAQMS Air-Quality Scraper
An air-quality study needed years of pollutant readings from a system that assumed a human would copy them out by hand. I wrote the scraper so nobody had to.
Stack
Python, pandas, HTTP APIs, Data Engineering
Links
Source withheld
Notes
It builds the encoded requests the CAAQMS system expects, pulls pollutant and environmental readings for every configured station, tracks which stations are done, and writes the results out to Excel for the actual analysis.
The numbers
- Collection
- Retrieved configured station data across 24 pollutant and environmental fields.
- Impact
- Eliminated more than 10,000 hours of manual entry for the research workflow.