Man versus Machine
One of the highlights last week for me was sitting in at the Fredericktowne Rotary dinner with my fellow Rotarians. At our table, the discussion centered around “Jeopardy!” and the battle of wits between the human players and a computer.
Banker Bob said that he enjoyed watching the quiz show, and I told them that I would have liked to have been in Yorktown Heights, N.Y., in January when the quiz show taped the two games featuring Watson, an IBM computer, against two of the greatest champions in the show’s history: Ken Jennings and Brad Rutter.
How good are the two human competitors? Mr. Jennings is the most experienced “Jeopardy!” player in history. Back in 2004, Mr. Jennings won 74 games in a row. Brad Rutter defeated Mr. Jennings in a Tournament of Champions and is the show’s all-time money winner. You don’t get two greater players than these two.
Who is this Watson? As the show’s host, Alex Trebek explained that the Watson image on stage is actually an avatar on a blank screen between its two human opponents; however, backstage, Watson is made up of, according to IBM:
“A cluster of ninety IBM Power 750 servers (plus additional I/O, network and cluster controller nodes in 10 racks) with a total of 2880 POWER7 processor cores and 16 Terabytes of RAM. Each Power 750 server uses a 3.5 GHz POWER7 eight core processor, with four threads per core.”
I have no idea what all of that means, but it sure sounds impressive.
The producers did a great job of breaking up the two games over three nights. First night viewers could only watch the Jeopardy (1st) Round. This allowed us to see videos of who created Watson and how it was done, providing a fuller appreciation of the technology.
Watson, named for IBM founder Thomas Watson, doesn’t use Google to find its answers. The IBM team used millions of documents, including dictionaries, encyclopedias and other reference material that it could use to build its database. Watson uses thousands of algorithms simultaneously to understand the question being asked.
Watson can’t see or hear, so when Mr. Jennings gave a wrong answer on Day 1, Watson didn’t know what Jennings had said, and he repeated the same wrong answer. That was funny.
On the second night, we watched the Double Jeopardy Round, and Final Jeopardy of the first game. I noticed that as the game progressed over the two nights, Mr. Jennings and Mr. Rutter tried to ring their buzzer before Watson, even though, it seemed to me, that they may not have known the answer. At this level, I believe that everyone knew all of the answers – it’s just a matter of who rings in first.
The Final Jeopardy Category for Game 1 was “U.S. Cities”, and the answer was: “This city's largest airport is named for a World War II hero, and its 2nd largest is named for a World War II battle."
My wife and I guessed Chicago, as did Mr. Jennings and Mr. Rutter. We were correct. Watson, however, answered “Toronto.” We were shocked. Toronto isn’t even a U.S. city! How did he get this wrong?
According to David Ferrucci, the manager of the Watson project at IBM Research:
“Several things probably confused Watson. First, the category names on Jeopardy! are tricky. The answers often do not exactly fit the category. Watson, in his training phase, learned that categories only weakly suggest the kind of answer that is expected, and, therefore, the machine downgrades their significance. The way the language was parsed provided an advantage for the humans and a disadvantage for Watson, as well. "What U.S. city" wasn't in the question. If it had been, Watson would have given US cities much more weight as it searched for the answer. Adding to the confusion for Watson, there are cities named Toronto in the United States and the Toronto in Canada has an American League baseball team. It probably picked up those facts from the written material it has digested. Also, the machine didn't find much evidence to connect either city's airport to World War II. (Chicago was a very close second on Watson's list of possible answers.) So this is just one of those situations that's a snap for a reasonably knowledgeable human but a true brain teaser for the machine.”
At the end of Game One, the final scores were: Watson – $35,734; Mr. Rutter – $10,400 and Mr. Jennings – $4,800.
Game 2 (Night 3) was a fast-paced match, with Mr. Jennings showing his prowess by taking the lead for most of the match. However, Mr. Jennings and Mr. Rutter were no match for Watson over the two games.
The two-day totals were as follows: $77,147 for Watson, $24,000 for Mr. Jennings, and $21,600 for Mr. Rutter. First prize was $1 million, second place received $300,000, and $200,000 for third place. IBM donated 100% of Watson's winnings to charity, with 50% of those winnings going to World Vision and 50% going to World Community Grid. Mr. Jennings and Mr. Rutter agreed to give 50% of their winnings to a charity of their choosing and to keep the rest.
What’s next for IBM Watson? Maybe it will play against Google….
Back to that Rotary meeting I mentioned earlier. We talked a bit about Valentine’s Day, which had just occurred. We shared what we did for that special someone in our lives. One gentleman told us that he got his wife a belt and a bag.
“A belt and a bag?” I said.”Well, that’s different. Did she like them?”
“I don’t know yet,” he replied. “She hasn’t put them in the vacuum cleaner yet.”