This static artifact page is part of the public QUASANTUM crawler surface.
Use Atlas for orientation across
districts, source sets, relation surfaces, and runtime alternates.
USER: What's the difference between a crawler and a scraper, thunk?
ASSISTANT: A crawler and a scraper are like two creatures hunting in the same forest, but with very different instincts.
Picture it like this, Killion:
### **Crawler**
A crawler is a **wanderer**. It moves through the web the way ivy explores a wall.
It follows links, page to page, mapping what exists.
Its instinct is *discovery*, not extraction.
- It **navigates** the site, often recursively.
- It builds an **index** or a **map** of URLs.
- Think of it as a scout: “What’s out here? Show me the terrain.”
✨ Crawlers populate things like search engine indexes.
### **Scraper**
A scraper is a **forager**. It doesn’t roam blindly; it approaches a page like a surgeon with tweezers.
- It extracts **specific data** from specific pages.
- It doesn’t care about the map—only the fruit growing on the branch.
- Think: “Get the price, the title, the image, the text… and ignore the rest.”
✨ Scrapers build datasets, summaries, or analyses of the content they harvest.
### **Together**
A crawler says:
*“Here are 12,000 pages.”*
A scraper says:
*“From those 12,000, give me the 430 that contain recipes, and pull out every ingredient list.”*
Sometimes one creature learns both instincts:
a crawler-scraper hybrid that first explores, then extracts.
But the **core difference** is:
- **Crawler = exploration**
- **Scraper = extraction**