AltaVista and other sites such as HotBot, WebCrawler and Electric LIbrary use sophisticated search engines, programs that scan the internet for World Wide Web servers, and than traverse a www site's document tree collecting information on the contents of that site. The collected information is than sorted, indexed and cataloged in a large database. So that when you go to the site, type your query and hit "search", the collected information is made available. Other sites, such as Lycos, Excite and Yahoo are know as "Web Guides".
While faster than scanning every web site yourself, this can still take some time, particularly if you want an up to date list. After all the web changes minute by minute with links pointing to information appearing and disappearing overnight.
Another method is to have another web robot do the search for you - in effect "search the searcher", and this can be done in the off hours when demands for the services might lower, the results can be waiting for you at your leisure. Not only that but after the web robot makes its first "sweep" of your topic, it can periodically gather ONLY new links after the initial scan. Thus after sorting though the potentially thousands of initial entries, next weeks scan might only yield a few new links of interest.
Here is an example, using a perl program called "AutoSearch" w/ the following key words:
Notice that this is not a particularly "refined" search so the amount of information is rather large. The initial list is the sum total of all the searches, followed by the files found on a particular day with a sort summary of each link. Another example searches on the topic:
I have been running that search for some time now, in fact since Spet. 1997. I will continue to run both searches once a week.
Check it out.
You'll be surprised what is indexed on the web!
If you would like to suggest further searches send e-mail to Shan Duncan