The “Internet IQ Evaluation Algorithm”
A new study from China has attempted to rank search engines by IQ, based on how well they answer a series of questions that typically appear on IQ tests.
The study, published in the Elsevier journal Procedia Computer Science, was presented in June at a conference in Russia.
The researchers tried out their new “search engine IQ test” on Google, Bing, and five popular Chinese search sites, and also gave the same test to three groups of children aged 6, 12, and 18.
Their test, which was written in Chinese, gave Google (in this case Google.com.hk) a slightly higher IQ than Bing, because it did better at translating and calculating. In the overall results, the highest IQ was achieved by So.com, followed by Baidu, Sogou, Google, Bing, Panguso, and Zhongsou.
The questions included common knowledge (“Which planet is the largest in the solar system?”) and basic arithmetic (“How much is 25 multiplied by 4″)? The search engines also had to rank a series of numbers from smallest to largest, and choose which word is out of place in the set “snake, tree, tiger, dog, rabbit.” They were also asked “How much is 1 + 1?” and told to answer in the form of text, audio, and image.
But none of the search engines came anywhere close to a 6-year-old child in terms of reasoning ability. The search engines outperformed humans in common knowledge, translation, and calculations, but did poorly in discovering patterns and making speculations.
For example, the engines were stumped when confronted with “20/5=4, 40/10=4, 80/20=4, 160/40=4: observe the rules, then design the fifth question,” or “If there are many animals in one place, but they are all in cages and many people are looking, then where is it?”
“The current abilities of search engines in these areas,” wrote the researchers, is “close to zero.”
Their research was partially in response to claims about artificial intelligence soon surpassing the human kind. Though that oft-prophesied moment of singularity has yet to arrive, the researchers hope their test will help chart the Internet’s intellectual development over time, and show how quickly (or slowly) the gap between human and machine intelligence is closing.
They admit that their test is still in the early stages, and thus imperfect. The main reason that Baidu outperformed Google, for example, was because Baidu has certain image-recognition features that the American search engines do not, which meant Bing and Google automatically failed some parts of the test.
“This is an ongoing research project,” wrote lead author Feng Liu, “which opens a door for us to predict how the future Internet will affect our society. We will report more systematic results in the near future.”