How I scraped 50+ Indian coffee roasters using AI tools, a PRD breakdown, and some stolen time during my anniversary vacation in Ooty
About 3 weeks ago, I launched IndianCoffeeBeans.com — a half-baked MVP built with caffeine, LLMs, and a deep fear of never shipping anything. You can read that chaotic build story here. Spoiler: ChatGPT basically bullied me into pressing publish.
But this post isn't about starting. It's about finishing… well, at least halfway. This was the backend phase. Frontend? Still a war zone.
Version 1: "Wait, Python is whitespace sensitive?"
I went in hot. Started writing everything from scratch — planned to scrape all Indian roasters, extract coffee details, enrich with LLMs, and fire it into a clean database.
Reality? TypeError: NoneType has no attribute 'find' · HttpUrl is not JSON serializable (What does that even mean?? Still don't know.)
I was tripping over broken type hints, misused dataclasses, and janky API calls like it was a landmine field. Eventually rage-quit. Deleted the whole thing.
Version 2: "The Linking Spiral of Doom"
Determined to be smarter, I rebuilt from scratch. Again. This time, I modularized everything: discovery, enrichment, upload, platform detection — 15 separate files.
Sounds smart, right? Until you realize you now have to share 15 files with GPT/Claude… only for them to lose context every damn time.
By the end, I was manually pasting imports, patching circular errors, and screaming into my keyboard. You guessed it. Nuked that repo too.
Version 3: "Okay, I'm gonna read the docs this time."
Third time, I humbled myself. I read the Crawl4AI docs (yes, actually). Discovered Taskmaster — a Claude-powered task runner that helped me break the PRD into manageable steps. Also paid $20 for Windsurf AI IDE, which — fine — was the best money I've spent on this project so far.
This time, I:
- Cleaned up my thinking
- Rebuilt the scraper pipeline block-by-block
- Wrote reusable schema-aware extractors
- Finally, finally got the damn thing running from
roaster_discoverer.pyall the way to Supabase upserts
And when I still got stuck? I hired a dev on Fiverr. Delegation is underrated.
Also… I was doing this from Ooty.
On my anniversary trip. With my wife and 4-year-old twins.
While they were off enjoying the hills, I was debugging an
awaitissue in a homestay with terrible Wi-Fi.Romantic? Debatable. Productive? Weirdly, yes.
Pro-tip: promise your wife you'll "just take 10 mins" to fix one bug… or better still, wait for everyone to sleep and then work. Next day is groggy, but totally worth it.
What I Finally Built
A working backend that:
- Scrapes 50+ Indian coffee roasters
- Extracts and enriches 200+ coffees
- Parses roast level, bean type, processing, flavor, brew method, and pricing
- Pushes clean data into Supabase
- Modular codebase with platform detection (Shopify/Woo/etc.)
- CLI pipeline that doesn't collapse under pressure
It's the cleanest, most functional backend I've ever shipped.
But here's the kicker: As a visual person, I still can't see anything that makes me feel it's good. Now I've got to wrestle with Next.js, Tailwind, and React — a different beast entirely.
What I Learned
- Rewriting sucks. But sometimes it's necessary.
- AI tools are great. But they're not miracle workers unless you structure your chaos.
- Don't be afraid to ask for help. Or pay for it.
- Don't build in a silo. Write the PRD. Break it. Validate it.
- And above all — finish something. Even if it's only half.
What's next?
- Frontend. Filters. Features. Hopefully, fun.
- Launching v2 in a couple weeks
- Inviting feedback, testers, roaster suggestions
- Dreaming of a roaster map + user-generated reviews
If you're out there, stuck between idea and execution hell — I feel you. Just keep showing up. Keep writing ugly code. Keep breaking things. Eventually, something will work. And if all else fails — debug on vacation.