Entry 10 / 12
Mar 2024
CAAQMS Web Scraping
Python scraper for India's CAAQMS air-quality network: JWT session handling, SQLite storage for portability, and 10,000+ hours of manual entry saved.
Stack
PythonSQLiteJWT
Links
Notes
A Python scraper for India's CAAQMS air-quality monitoring data. The portal sits behind a JWT-based session with a finicky token-refresh loop, so the bulk of the work was the auth dance, not the data shaping. SQLite for portability - research collaborators could sync the file rather than spin up a database. Saved the team well over 10,000 hours of manual entry across its lifetime.