Crawlers (or bots) are used to collect information out there on the web. By utilizing web site navigation menus, and studying inner and exterior links, the bots begin to grasp the context of a web page. Of course, the words, photographs, and different data on pages also help search engines like goog