Apple’s Siri Gets a ‘D’ Grade, Google Search Gets ‘B+’ in 1600 Question Test

Piper Jaffray’s Gene Munster tested Apple’s Siri with Google’s Search by firing 1600 questions at each of them.

Not surprisingly, Google’s search came out on top, answering correctly 86% of the time, whereas Siri managed to answer only 68% correctly.

Munster asked 800 questions on the busy streets of Minneapolis, 800 in a quiet room. He has published the results:

  • Google understands 100% of the questions (not surprisingly, since they are keyed in)
  • Google replies accurately 86% of the time
  • Siri comprehends 83% of queries in noisy conditions, 89% in a quiet room
  • Siri answers accurately 62% of the time on the street and 68% in a quiet room.

Munster gave Siri a ‘D’ grade and Google search a B+. Munster writes:

“In order to become a viable mobile search alternative. Siri must match or surpass Google’s accuracy of B+ and move from a grade D to a B or higher.” 

According to Munster, Siri is at least two years behing Google Search, but expects Siri to improve in iOS 6 and has the potential of being disruptive in the future. He also expects Apple to reduce its reliance on Google from 60% to 48%. He breaks down Siri’s reliance on Google further:

“Google provides 100% of navigation results, 61% of information results, 48% of commerce results and 42% of local results. Among other result aggregators, Yelp provided the most local results (51%) and commerce results (51%), while WolframAlpha provided 34% of information results.”

Image credit: Gizmodo

Munster also gave examples of the type of errors Siri was making:

  • What team does Peyton Manning play for? Responded with the answer to the previous query. This was the most common error.
  • Where is Elvis buried? Responded I can’t answer that for you. It thought the persons name was Elvis Buried.
  • Where am I? Pin dropped in the wrong place.
  • When did the movie Cinderella come out? Responded with a movie theater search on Yelp.
  • How do I get from Boston to New York? Responded I can only give directions from your current location. I cant give you directions to a place you are not in.
  • What spices are in Lasagna? Responded with a Yelp search with lasagna on the menu.
  • When is the next Haley’s comet? Responded “You have no meetings matching Haley’s”
  • I want to go to Lake Superior? Responded with directions to the company Lake Superior X-Ray.

It may be a little unfair to compare Siri with Google Search, but that’s the benchmark Apple needs to match or surpass Google’s a#curacy, if it wants to Siri to be truly disruptive.

[via CNNMoney]