Why Watson Thought Toronto was a US City

In an interview over at The Browser, Stephen Baker, who has a new book on IBM's Watson and computer intelligence, explains how the computer could have possibly thought Toronto was a US city:

Let's review this. TheJeopardy! category was U.S. Cities, and the clue was, 'This city's largest airport is named after a World War II hero, and its second largest airport after a World War II battle.' Watson starts hunting around, looking for these two connections to airports throughout the U.S. and, more broadly, North America and the world. Why would it look beyond the U.S.? Because Watson is never completely sure that it understands the clue. It has to hedge a bit, and allow for the fact that it might not understand.

Watson has also learned, through statistical analysis of the Jeopardy! categories, that they don't always coincide with the question. For example, a clue on American novelists might say, 'This masterpiece features a young man named Holden Caulfield.' The answer to the clue is not J D Salinger, it's Catcher in the Rye. Watson is aware - statistically at least -- that categories can't always be trusted.

So in the U.S. cities/airport question, Watson goes on a hunt and never really finds an answer it has high confidence in. It has abysmal confidence in both Toronto, which has a couple of airports named after World War I heroes, and Chicago. It probably doesn't understand the Battle of Midway, so it doesn't make that connection. Because it has very low confidence, it doesn't rule out Canada. A lot of people would say, 'Well, that's a sign of idiocy,' and you could argue that, in this case, it was. But Watson has to allow for exceptions.

The whole interview, which is actually about other books around the topic of artificial intelligence is worth a read. Also, on the topic of Watson, Stephen Wolfram (of WolframAlpha) has an interesting take.