Programming Spiders, Bots, and Aggregators in Java

A tremendous amount of information is available through the Internet: today’s news, the location of an expected package, the score of last night’s game, or the current stock price of your company. Open your favorite browser, and all of this information is only a mouse click away. Nearly any piece of current information can be found online; you have only to discover it.
Most of the information content of the Internet is both produced and consumed by human users. As a result, web pages are generally structured to be inviting to human visitors. But is this the only use for the Web? Are human users the only visitors a website is likely to accommodate?