Despite the rumors, Google’s Pathways Language Model (PaLM), which was unveiled earlier this year, continues to dominate the AI field.
PaLM can be scaled up to 540 billion parameters, which means that the performance across tasks keeps increasing with the model’s increasing scale, thereby unlocking new capabilities. GPT-3, in contrast, has only about 175 billion parameters. Tens of thousands of Google’s own TPU (Tensor Processing Unit) chips can be used to scale a model using Pathways.
Furthermore, PaLM outperformed previous large models like GPT-3 and Chinchilla on 28 out of 29 NLP tasks, outperforming the majority of cutting-edge benchmarks as well as the average human.
Read More: Google’s PaLM is Ready for the GPT Challenge
