A web crawler (or spider) is an internet bot that systematically browses the web to index content for search engines.