Statistical Modeling, Causal Inference, and Social Science

There has been a pile of discussion over the past few years regarding the use of data mining technology in politics. The public perceives data mining as an intrusion of their privacy. As with any kind of technology, it is hard to restrict it: it just goes underground. At the same time, new technology used improperly is just like a new gun to a soldier - in the short run he's better off with a better gun, but in the long run everyone's worse off because the enemies will have better guns too. One answer to the efforts to data mine from the top is to data mine from the bottom too. The technology does enable the administration to observe the population, but at the same time the population can use the technology to observe the actions of the administration. While we hear about the top-down efforts from the media, there are several interesting bottom-up efforts going on, and they demonstrate the possible uses of technology.

John Walker built a hyperlinked version of parts of the US Code, and identified interesting special provisions, which would probably remain hidden without technology. GovTrack.us collects information about the US Government, linking it with discussions. There are several other web sites with a similar role: http://www.vote-smart.org/ and http://www.opensecrets.org/. PoliticalFriendster maintains profiles of officials along with their connections. The one that seems to have started this all, was MIT's Open Government Information Awarness, now accessible only through the web archives. OGIA was itself inspired by DARPA's Information Awareness Office, also only accessible through web archives. We have done some experiments with techniques for accessing the roll call voting records in a slightly more accessible way - and it is fascinating that some of the proximities that appear in the PoliticalFriendster also appear in the roll call vote record. Finally, one can view the structure of the budget graphically too.

So, in conclusion, technology is affecting how politics is done. The distinct asymmetry of mass broadcasting media differs from a somewhat more symmetric World Wide Web.