crawlable
Accessible for exploration
Crawlable describes content that can be indexed by web crawlers—the automated programs search engines use to discover and catalog pages.
A crawlable site has proper structure, working links, and no barriers preventing bots from reading it: Make sure your new pages are crawlable before launch.
The opposite is content hidden behind logins, blocked by robots.txt, or buried in formats crawlers can't parse.
Web-spinning arachnid
- botAutomated software agent
- sitemapWebsite's page directory
- robots.txtCrawler directive file
- SEOImproving site discoverability
metadata · canonical · hyperlink · anchor · cache · backlink · indexer · domain
Sloped connecting surface
- wheelchairSeated conveyance for mobility-impaired
- mobilityCapacity for movement
- barrier-freeWithout physical obstructions
- handrailBar for gripping on stairs
assistive · adaptation · accommodation · step-free · reachable · ground-level
spider
Web-spinning arachnid
- botAutomated software agent
- sitemapWebsite's page directory
- robots.txtCrawler directive file
SEO · metadata · canonical · hyperlink · anchor · cache · backlink · indexer · domain · fetch
ramp
Sloped connecting surface
- wheelchairSeated conveyance for mobility-impaired
- mobilityCapacity for movement
- barrier-freeWithout physical obstructions
handrail · assistive · adaptation · accommodation · step-free · reachable · ground-level
crawlableblocked
Passage prevented or impeded
hidden · inaccessible · restricted · disallowed · private
crawlableimpassable
Impossible to traverse
untraversable · blocked · obstructed · impenetrable · inaccessible