I needed 100,000 product descriptions for a classification model. Budget for data labeling: zero.

Web scraping saved the project. But it also taught me that scraping for ML has its own set of challenges.

Data quality from scraping is wildly inconsistent. HTML structures change between pages. Missing fields are common. Duplicates are everywhere. And the data reflects the biases of whatever websites you scraped.

My scraping-for-ML pipeline: scrape broadly first (get everything), then clean aggressively (remove duplicates, fix encoding, standardize formats), then validate (check for completeness and consistency), then sample and manually inspect 200-300 examples to verify quality.

Tools: Scrapy for structured crawling, BeautifulSoup for HTML parsing, trafilatura for article extraction. For JavaScript-heavy sites, Playwright.

Legal considerations matter. Check robots.txt. Respect rate limits. Understand the terms of service. And if you're building a commercial product, consult a lawyer about the data's licensing.

The meta-lesson: the best ML training data often comes from creative data sourcing, not expensive labeling contracts. Existing databases, public datasets, web scraping, synthetic augmentation — stack these together and you can build competitive models on a startup budget.

Resourcefulness > resources when it comes to data.