Baichuan Intelligence, a startup founded by Wang Xiaochuan, the creator of Sogou and a prominent figure in the Chinese tech industry, has unveiled its latest achievement: Baichuan-13B, a cutting-edge large language model (LLM). With aspirations of developing China’s own version of OpenAI, Wang has made significant progress toward his goal.
Wang, known for his impressive background as a computer science prodigy from Tsinghua University, departed from Sogou in late 2021. In April, he launched Baichuan Intelligence, quickly securing $50 million in financing from a group of angel investors as the hype surrounding ChatGPT, OpenAI’s language model, swept the globe.
Baichuan is gaining recognition as one of China’s most promising LLM developers, leveraging its founder’s expertise and reputation. Similar to other homegrown Chinese LLMs, Baichuan-13B is built on the Transformer architecture, the foundation for models like GPT, and is trained on Chinese and English data. The model, available as open-source, is specifically optimized for commercial applications, as stated on its GitHub page.
Impressively, Baichuan-13B, boasting 13 billion parameters, is trained on an extensive 1.4 trillion tokens. In comparison, Meta’s LLaMa, another large language model, utilizes 1 trillion tokens in its 13 billion-parameter version. Wang previously expressed his commitment to releasing a large-scale model that rivals OpenAI’s GPT-3.5 by the end of this year.
Despite being a relatively young company, Baichuan has made remarkable strides in its development. By April’s end, the team had already grown to 50 members, and in June, they launched their first LLM, Baichuan-7B, which features 7 billion parameters.
Baichuan-13B
The foundational model, is now accessible to academics and developers who have obtained official approval to utilize it for commercial purposes, and notably, it offers variations that can operate on consumer-grade hardware, including Nvidia’s 3090 graphic cards. This flexibility is particularly valuable considering the chip sanctions imposed by the U.S. on China.
Several other Chinese companies have also made significant investments in large language models. Baidu, the search engine giant, has dedicated resources to this technology, while Zhipu.ai, a spinoff of Tsinghua University led by Professor Tang Jie, and the research institute IDEA, led by Harry Shum, a co-founder of Microsoft Research Asia, are actively involved in LLM research and development.
China’s pursuit of large language models aligns with its preparations to implement stringent AI regulations. The country is expected to introduce regulations specifically focused on generative AI, emphasizing content control. These measures indicate an intensified level of oversight compared to the rules introduced in April. In addition, companies may be required to obtain licenses before launching large language models, which could potentially slow down China’s efforts to compete with the United States in this burgeoning industry.
