Please note that the value of this tag is considered a hint and not a command. Even though search engine crawlers consider this information when making decisions, they may crawl pages marked “hourly” less frequently than that, and they may crawl pages marked “yearly” more frequently than that. It is also likely that crawlers will periodically crawl pages marked “never” so that they can handle unexpected changes to those pages.
November 8th, 2008 |
Tags: Search Engine Crawling
Home