Piper Jaffray’s Gene Munster tested Apple’s Siri with Google’s Search by firing 1600 questions at each of them.
Not surprisingly, Google’s search came out on top, answering correctly 86% of the time, whereas Siri managed to answer only 68% correctly.
Munster asked 800 questions on the busy streets of Minneapolis, 800 in a quiet room. He has published the results:
- Google understands 100% of the questions (not surprisingly, since they are keyed in)
- Google replies accurately 86% of the time
- Siri comprehends 83% of queries in noisy conditions, 89% in a quiet room
- Siri answers accurately 62% of the time on the street and 68% in a quiet room.
Munster gave Siri a ‘D’ grade and Google search a B+. Munster writes:
“In order to become a viable mobile search alternative. Siri must match or surpass Google’s accuracy of B+ and move from a grade D to a B or higher.”
According to Munster, Siri is at least two years behing Google Search, but expects Siri to improve in iOS 6 and has the potential of being disruptive in the future. He also expects Apple to reduce its reliance on Google from 60% to 48%. He breaks down Siri’s reliance on Google further:
“Google provides 100% of navigation results, 61% of information results, 48% of commerce results and 42% of local results. Among other result aggregators, Yelp provided the most local results (51%) and commerce results (51%), while WolframAlpha provided 34% of information results.”
Image credit: Gizmodo
Munster also gave examples of the type of errors Siri was making:
- What team does Peyton Manning play for? Responded with the answer to the previous query. This was the most common error.
- Where is Elvis buried? Responded I can’t answer that for you. It thought the persons name was Elvis Buried.
- Where am I? Pin dropped in the wrong place.
- When did the movie Cinderella come out? Responded with a movie theater search on Yelp.
- How do I get from Boston to New York? Responded I can only give directions from your current location. I cant give you directions to a place you are not in.
- What spices are in Lasagna? Responded with a Yelp search with lasagna on the menu.
- When is the next Haley’s comet? Responded “You have no meetings matching Haley’s”
- I want to go to Lake Superior? Responded with directions to the company Lake Superior X-Ray.
It may be a little unfair to compare Siri with Google Search, but that’s the benchmark Apple needs to match or surpass Google’s a#curacy, if it wants to Siri to be truly disruptive.
[via CNNMoney]
