Baidu claims Ernie 3.5 beat GPT 4 in Chinese language tests and “comprehensive ability scores.”
Chinese tech major Baidu claimed on Tuesday that its artificial intelligent model Ernie outperformed OpenAI’s widely popular ChatGPT in some key tests. According to media reports, Baidu’s latest AI model is claimed to not only be more capable than the previous versions but also performed better than OpenAI’s GPT 3.5 model and even beat the latest and more advanced GPT 4.
Citing a recent test conducted by the China Science Daily journal, Baidu said Ernie 3.5 beat GPT 4 in Chinese language tests and “comprehensive ability scores.” Talking about one such test on standard admission and qualification exams, such as those required to qualify for college or law school, Baidu’s Ernie 3.5 outperformed both GPT-3.5-based ChatGPT and GPT 4 models in Chinese. However, the chatbot lagged behind GPT 4 in the English language even though it was ahead of ChatGPT in another test released by OpenAI’s backer Microsoft.
In another test containing over 13,000 multiple-choice questions covering over 50 different subjects in Chinese, Ernie 3.5 scored higher than ChatGPT and GPT 4. However, in a third test developed by a group of U.S. universities that comprised various subjects like science and humanities, Ernie 3.5 was behind both ChatGPT and GPT 4.
The company’s CTO Haifeng Wang said that Baidu’s latest model has achieved “broad enhancements in efficacy, functionality, and performance” and these improvements are more capable than previous versions in several tasks like creative writing, Q&A, reasoning, and code generation. Wang added that the one defining feature of the latest AI model Ernie 3.5 is that it is not able to access “plugins” which are apps that can be accessed via Ernie Bot. Baidu’s search feature has been integrated with Ernie Bot via a plugin, for example.
Meanwhile, in April, Baidu filed lawsuits against Apple and “relevant” app developers over a counterfeit version of its artificial intelligence bot. The Chinese company reportedly filed lawsuits in the Beijing Haidian People’s Court. Baidu then said in a statement that its AI bot does not have an official app at the moment and until the tech company makes an official announcement of the same, any Ernie apps on the App Store are fake.
It is noteworthy that Baidu’s Ernie bot is currently available only to those who apply for and receive codes and the Chinese company, which is popularly regarded at the country’s Google, has further warned against people selling access codes. In March, Baidu unveiled its AI chatbot as a competition to OpenAI’s massively popular ChatGPT.
On March 16, Baidu chief executive officer Robin Li presented a pre-recorded video wherein Ernie Bot presented features like mathematical logic reasoning, generated a conference poster and video based on a prompt, and answered questions related to Chinese fiction, among others. Later that month, Baidu unveiled new capabilities of the AI bot in a closed-door meeting.
The advances made by Baidu’s Ernie underscore the intensifying AI race following the launch of OpenAI’s ChatGPT in November last year. Short for Chat Generative Pre-trained Transformer, ChatGPT is a popular AI text generator and has been a center of attention for many due to its human-like response. Several other tech giants, including Google with its AI bot Bard, have also started investing heavily in the segment.
