Entry 07 / 09
Mar 2024
CAAQMS Web Scraping
Air quality data scraper using SQLite and JWT, saving 10,000+ hours.
PythonSQLiteJWT
A Python scraper for India's CAAQMS air-quality monitoring data. The portal sits behind a JWT-based session with a finicky token-refresh loop, so the bulk of the work was the auth dance, not the data shaping. SQLite for portability - research collaborators could sync the file rather than spin up a database. Saved the team well over 10,000 hours of manual entry across its lifetime.