Terrorists don’t use the Internet solely to recruit members, spread their ideology, and raise funds for their activities. They also use it to conduct their own internal debates, creating a rich pool of information for analysis by counterterrorist groups. Gabriel Weimann, a professor of communication at the University of Haifa, Israel, and author of the book Terror on the Internet monitored just 12 Web sites operated by terrorist groups in 1998. Today he monitors 5,800.

        Intelligence agencies are having trouble keeping up with the volume. That’s why researchers from the Artifi cial Intelligence Lab at the University of Arizona have developed a set of automated tools to collect and analyze terrorist content on the Internet in a systematic way. The project, named the Dark Web, uses Web spidering to fi nd and catalog millions of Web pages, postings to terrorist forums, videos, and other multimedia content. The Dark Web has identifi ed seven jihadist Web sites that host 90 percent of the information related to improvised explosive devices, such as instruction manuals and videos, says project director Hsin chun Chen. The fi ndings are passed on to military intelligence agencies.

        “The first version of our spider could only collect 10 to 15 percent of the content. Now we can get about 85 to 95 percent,” says Chen.

          proxies and randomized processes so that they don’t get blocked. The group has also developed analysis tools to make sense of the content and prioritize the information. For example, “authorship analysis” captures the writing styles of anonymous senders, and “sentiment analysis” identifies who on the Web is the most violent and radical. “Some forums have a quarter million people posting, so there is no way anyone can eyeball those results,” explains Chen. “[Now] we can analyze millions of postings in a matter of seconds.”
           Another independent group participating in the monitoring is the Virginia-based Terrorism Research Center. Its deputy director, Ned Moran, says that tools such as the Dark Web help intelligence analysts do the first pass. “They take this massive entity of the Internet and scale it down into something more manageable.” But Moran retains some skepticism about the ability of such automated tools as spiders to differentiate adequately between relevant data and junk.

           Weimann agrees with Moran that human judgment remains an invaluable tool in determining which sites contain potentially harmful content. “What automated search can do is save expenses, manpower, and time. But it is limited in the depth of the analysis. Human eyes and mind see more and deeper than a crawler,” he says.—Isabelle Groc