A web crawler (or spider) is a program that browses the internet and indexes web content for search engines.