I got a report that Kaffenum's products were all showing as discontinued on IndianCoffeeBeans.com. But I could go to their Wix store right now and see every single one. The scraper was running fine. No errors. No crashes. No failed API calls. Just nine products marked missing in the first step of a two-step kill chain. One more scraper run and they'd be gone.
The debugging session that followed uncovered two independent bugs, a near-miss that would have permanently deleted 16 specialty coffees — including a Gesha and a Sudan Rume — and a lesson about data pipelines that I apparently needed to learn the hard way. For the fourth time.
Layer 1: The Symptom
The scraper was doing exactly what it was told. Products not seen in the latest fetch get marked missing. Next run, missing products get marked discontinued. Simple. So why were Kaffenum's products — still live, still selling — being marked missing?
I queried the database. Compared raw_products rows against the current Wix fetch. That's when I noticed the IDs didn't match.
The current scraper uses UUIDs: 696a5a11-a80f-ee39-157b-969d7f3cf098. These rows had something else entirely: 0b130231c92fa734. 16-character hex. The scraper has never produced a hex ID. So where did these come from?
The smoking gun: all 37 hex-ID rows had first_seen and last_seen set to April 15, 2026. The exact date I migrated from n8n to Python.
Layer 2: The Migration Artifacts
These weren't bugs. They were ghosts.
The old n8n workflow used hex IDs as platform_product_id. The new Python scraper uses Wix UUIDs. The matching logic in router.py keys on platform_product_id — hex ≠ UUID, so every legacy row appeared to have "disappeared from the store." Three Wix roasters affected: Kaffenum (9), GB Roasters (21), Anecdote (7). 37 orphan rows. All from a system that stopped running three months ago.
The fix seemed straightforward: delete the orphan rows. They're duplicates of UUID rows that already exist. Right?
Layer 3: The Cascade I Almost Didn't Check
Before writing the DELETE, I checked the foreign key cascade. raw_products → coffees (CASCADE) → favorites, curation data (CASCADE). If I delete a raw_product row, its coffee row gets deleted too. And if that coffee row gets deleted, every user favorite and curation reference to it vanishes.
So I queried: which HEX16 coffees have no UUID twin?
Sixteen rows.
Dawnspire. Hydro. Pure Bliss. Servarayan. Gesha. Sudan Rume. Monsooned Malabar. These weren't duplicates. They were the only copy of these coffees in the database.
Layer 4: Why the New Scraper Never Saved Them
The current scraper did fetch these products. It just classified them as is_coffee=False and never created a coffees table row.
Root cause: Wix stores product descriptions in additionalInfo[].description — an HTML blob nested inside an array. The top-level description field? Empty. split_wix_product mapped body_html from the empty field. The LLM sees a product title with no description and concludes: "this is not coffee."
The old n8n scraper got this right. The new Python scraper got it wrong. And the only reason we still had this data at all was because n8n created the HEX16 rows before I migrated — rows the new scraper couldn't match and couldn't recreate. A double failure that was, ironically, preserving the data.
The Fix
Two bugs, one afternoon:
- Bug #1 (migration artifacts): Migrate HEX16 rows to UUIDs where twins exist. Reclassify the 16 orphan coffees through the corrected pipeline. Then clean up the dead duplicates.
- Bug #2 (Wix field mapping): Fix
split_wix_productto readadditionalInfo[].descriptioninstead of the empty top-level field. Re-run classification on affected products.
What I Learned (Again)
I've now hit this pattern four times on ICB: scrapers oscillating terminal states, PostHog anonymous users collapsing into each other, metadata silently returning null, and now migration ghosts conspiring with field mapping bugs to nearly delete real data. Same disease, different organs.
The debugging discipline is what caught it: symptoms → root cause → scope → cascading effects. If I'd stopped at Layer 1 and run that DELETE script, I would have lost 16 specialty coffees — including varietals with no other data source. The second bug was invisible until I checked what the first bug's fix would destroy.
Before you run that cleanup script, ask: what else will this break? And check the cascade. Always check the cascade.