Articles with PalM 2

Google has commenced its annual I/O conference with a strong emphasis on advancing artificial intelligence (AI) across its various domains, with a particular spotlight on PaLM 2.

Google I/O has traditionally served as a developer conference, covering a wide range of topics. However, this year’s event stands out as AI takes center stage in almost every aspect. Google aims to establish itself as a frontrunner in the market, even as competitors like Microsoft and OpenAI enjoy the success of ChatGPT.

The cornerstone of Google’s endeavors is its newly introduced PaLM 2, a large language model (LLM). PaLM 2 will provide the backbone for at least 25 Google products and services, which will be extensively discussed in sessions at I/O. These include Bard, Workspace, Cloud, Security, and Vertex AI.

Originally launched in April 2022, the initial version of PaLM (Pathways Language Model) served as Google’s foundational LLM for generative AI. According to Google, PaLM 2 significantly enhances the company’s generative AI capabilities in meaningful ways.

During a roundtable press briefing, Zoubin Ghahramani, VP of Google DeepMind, emphasized Google’s mission to make information universally accessible and useful. He highlighted how AI has accelerated this mission, providing opportunities to gain a deeper understanding of the world and create more helpful products.

As Google showcases PaLM 2 and its far-reaching implications at the I/O conference, it solidifies its commitment to advancing AI and harnessing its potential to improve user experiences and product functionality.

Putting state-of-the-art AI in the ‘palm’ of developers’ hands with PaLM 2

Ghahramani explained that PaLM 2 is a state-of-the-art language model that is good at math, coding, reasoning, multilingual translation and natural language generation. 

He emphasized that it’s better than Google’s previous LLMs in nearly every way that can be measured. That said, one way that previous models were measured was by the number of parameters. For example, in 2022 when the first iteration of PaLM was launched, Google claimed it had 540 billion parameters for its largest model. In response to a question posed by VentureBeat, Ghahramani declined to provide a specific figure for the parameter size of PaLM 2, only noting that counting parameters is not an ideal way to measure performance or capability.

Ghahramani instead said the model has been trained and built in a way that makes it better. Google trained PaLM 2 on the latest Tensor Processing Unit (TPU) infrastructure, which is Google’s custom silicon for machine learning (ML) training. 

PaLM 2 is also better at AI inference. Ghahramani noted that by bringing together compute, optimal scaling and improved dataset mixtures, as well as improvements to the model architectures, PaLM 2 is more efficient for serving models while performing better overall.

In terms of improved core capabilities for PaLM 2, there are three in particular that Ghahramani called out:

Multilinguality: The new model has been trained on over 100 spoken-word languages, which enables PaLM 2 to excel at multilingual tasks. Going a step further, Ghahramani said that it can understand nuanced phrases in different languages including the use of ambiguous or figurative meanings of words rather than the literal meaning.

Reasoning: PaLM 2 provides stronger logic, common sense reasoning, and mathematics than previous models. “We’ve trained on a massive amount of math and science texts, including scientific papers and mathematical expressions,” Ghahramani said.

Coding: PaLM 2 also understands, generates and debugs code and was pretrained on more than 20 programming languages. Alongside popular programming languages like Python and JavaScript, PaLM 2 can also handle older languages like Fortran.

“If you’re looking for help to fix a piece of code, PaLM 2 can not only fix the code, but also provide the documentation you need in any language,” Ghahramani said. “So this helps programmers around the world learn to code better and also to collaborate.”

PaLM 2 is one model powering 25 applications from Google, including Bard

Ghahramani said that PaLM 2 can adapt to a wide range of tasks, and at Google I/O the company has detailed how it supports 25 products that impact just about every aspect of the user experience.

Building off the general-purpose PaLM 2, Google has also developed the Med-PaLM 2, a model for the medical profession. For security use cases, Google has trained Sec-PaLM. Google’s ChatGPT competitor, Bard, will now also benefit from PaLM 2’s power, providing an intuitive prompt-based user interface that anyone can use, regardless of their technical ability. Google’s Workspace suite of productivity applications will also get an intelligence boost, thanks to PaLM 2.

“PaLM 2 excels when you fine-tune it on domain-specific data,” Ghahramani said. “So think of PaLM 2 as a general model that can be fine-tuned to achieve particular tasks.”