A common aphorism among artificial intelligence practitioners is that A.I. is whatever machines can’t currently do.
Adam Gopnik, writing for the New Yorker, has a review called Get Smart in the most recent issue (4 April 2011). Ostensibly, the piece is a review of new books, one by Joshua Foer, Moonwalking with Einstein: The Art and Science of Remembering Everything, and one by Stephen Baker Final Jeopardy: Man vs. Machine and the Quest to Know Everything (which would explain Baker’s spate of Jeopardy!-related blog posts). But like many such pieces in highbrow magazines, the book reviews are just a cover for staking out a philosophical position. Gopnik does a typically New Yorker job in explaining the title of this blog post.
Gopnik describes his mother as “a logician, linguist, and early Fortran speaker” and goes on to add that she worked on an early machine translation project in Canada. I’m guessing she’s the Myrna Gopnik behind this 1968 COLING paper (LE PROJET DE TRADUCTION AUTOMATIQUE A L’UNIVERSITE DE MONTREAL).
As a child, Gopnik’s mom explained to him why a tic-tac-toe-playing computer is not very smart. I also had a run-in with an early tic-tac-toe computer at a science museum that left a deep impression on me. I wanted to build one in 6th grade, but only got as far as writing the game tree out on two sheets of posterboard, being stumped at how to code the game tree on my Digi-Comp, the instruction manuals for which described game trees clearly enough a 10-year old could understand them.
Coincidentally, John Prager just gave a very insightful talk today on the natural language processing (as well as strategy and hardware) behind IBM’s Watson system for playing the televised quiz show Jeopardy!.
I was hugely impressed with Watson. It answered almost every question correctly and grammatically. As Prager pointed out, some of the language is hard. Some of the categories (like “Rhyme Time”) had special components built for them, but most were totally general.
Given the buzz-in versus precision graphs Prager showed, I’m guessing Watson might not have won without the buzzer advantage. That’s because the humans Watson was playing had a history of getting almost every question right. Empirically, the champions answered roughly 70% of the questions they faced and answered 90-100% of them correctly. And that’s with competition to the buzzer from the other players. Watson’s ROC-type curve showed them in the winner’s territory without competition to the buzzer. My former CMU colleague Peter Spirtes was a Jeopardy! winner and he said it was very much about the buzzer because everyone was so good.
There would be two ways to make the questions harder. By making the underlying data more obscure, it’d probably favor Watson. By making the wordplay more obscure, I’m still voting on the human contestants.
Coming back to the over-arching subject of this blog, Prager emphasized the classification component of the system that used hundreds of features of a question, the category and a possible answer to estimate the probability that the answer is correct. Exactly that same opinion is what led me into Bayesian stats, where characterizing uncertainty is the name of the game.
I was also very impressed by Watson. I had no idea we were anywhere near advanced enough to even be competitive at that level.
With regard to the post title… that is I suppose true due to the failure to create true A.I.. After the mid-eighties (I'm told at least as I was 4) it became very apparent that the quest to create a general reasoning computer had failed, after which A.I. was redefined to include any kind of inference. I sort of wish that the term remained pure, and referred to a turing test passing machine.
People tend to be especially surprised when computers perform in ways that _they_ view as instinctual or emotional. When I created my Texas Hold'em poker playing program based on a Nash equilibrium, (non-math) people were very surprised that it bluffed because they viewed bluffing as an instinctual mind game with their opponent. I explained that bluffing was a necessary part of the solution to the game because the game was not just about the cards, but rather also about the relative information that each player has. If the computer never bluffed, then that would give the opponent more information about the cards that it held, and lead to tactical disadvantage.
Ian:
I knew a guy in grad school–he was studying chemistry–who refused to believe there was any skill in poker. He insisted that it was all luck, and that all that talk of poker hustlers etc. was just b.s. I was unable to convince him otherwise.
I find AI frustratingly hard to define. Is Bayesian classification AI? How about genetic algorithms or neural networks. Is any kind of program that appears "smart", AI?
I think the big difference between A.I. in the 1970s and 1980s and now is that we've reached equilibrium of the promise versus delivery.
In the 1970s and 1980s, everyone talked about big A.I., which consisted of human-like reasoning. There were projects like Cyc to try to organize all of human knowledge in a structured fashion. And neural networks were being pitched as models of human cognition, not as non-linear classifiers.
Nowadays, we know we're building classifiers with some clever search, fielded examples in NLP being systems like Watson or Google Translate. Neither of these systems does anything like the reasoning we were trying to model in the 1980s.