It might be support the long-term mission of Wikipedia to close the site to robots, or perhaps specifically to Googlebot. Although Wikipedia's high internet profile brings numerous advantages, the disadvantages are arguably of greater importance.
Advantages of a high profile
edit- Wikipedia attracts a large number of interested users, many of whom make constructive edits
- People are able to actually find information using a search engine, rather than first having to find Wikipedia, and then doing a new search (using the poor search engine available) in order to find information.
Disadvantages of a high profile
edit- Wikipedia attracts a large number of interested users, many of whom make malicious or careless edits
- Wikipedia becomes an inviting target for POV-pushers, who feel that through changing Wikipedia entries they can easily reach a wide internet public. (see recent Talk:Dokdo RM poll, in which an entire country was actively recruited to serve as meatpuppets)
- Wikipedia becomes an inviting target for spammers and vandals, who can readily achieve visibility here.
- Wikipedia users become focused on short-term presentability rather than long-term development of articles, leading to an unwholesome focus on having "pretty" or "professional-looking" articles when these articles are full of substantial gaps and errors.
- Corollary: work that is most constructively focused on release versions is diverted to the live wiki instead.
Advantages of indexing in general
edit- Cached versions of pages are available even when servers are down or slow.
- Offsite search services (notably Google and Yahoo) can be used. This is necessary when the onsite search function is disabled (as it often is), and is frequently convenient even when the search function is working properly.
Disadvantages of indexing in general
edit- Cached versions of deleted pages are also readily available, potentially compounding legal concerns.