Tuesday, December 19, 2006

Data Mining dead for effective counterterrorism?

Jeff Jonas has just posted one paper in his blog saying that data mining would fail to detect suspicious for counterterrorism. This kind of voice is really interesting to me.

In his article, he mentioned the limitations of existing data mining techniques:

1. typical data mining requires lots of data available to discover the pattern of terrorism. However, terrorism's frequency is so low that data mining would fail to detect it.

2. the cost of false positive. If the number of population is large, suppose 1 billion people under monitored, even with a very high accuracy(say 99.9%), the false positive would result in 1,000,000 suspicious, which would be too costly to perform furthur investigation and tracking.

After reading it, I just have several questions in mind:
  1. Is inductive data mining dead for this case?
  2. Is deductive analysis more suitble for this case?
  3. How to detect novelty? What is an "anomly"?
  4. Terrorists will change their stratgy for next attack. Is it possible to find it?

Instead of claiming the limitedness of data mining, I would cheer for the "dead" of statistical data mining. Machine learning, finally have to rethink about its correct direction.

No comments: