Entry 09 / 12
Mar 2024
Google Scholar Scraping
Automated Google Scholar citation scraper with careful pacing, resumability, and dedup, freeing 520+ hours of manual citation entry for an NUS research lab.
Stack
PythonPostmanAutomation
Links
Notes
Built to free 520 hours the lab was burning on manual citation entry. Scholar doesn't expose an official API, so the tool routes through Postman with careful pacing to avoid getting flagged. Boring code that solved a real problem - most of the value was in the resumability and the dedup on the backend, not the scrape itself. A small reminder that the unglamorous infrastructure projects often return the most time.