In a recently leaked document originating from a researcher within Google, the author raises concerns about the growing threat of open-source AI models to Google’s dominance in the field. The text, shared anonymously on a public Discord server, presents the researcher’s personal opinions, and not the views of Google as a whole. The document highlights the rapid advancements in open-source AI, particularly in large language models (LLMs) and image generation, and proposes that Google should learn from and collaborate with the external community.
The document cites several examples where open-source AI has overtaken traditional models:
1. LLMs on a Phone: Users can now run foundation models on a Pixel 6 at 5 tokens/sec.
2. Art Models: Websites offer unrestricted access to art models, with text models following closely.
3. Multimodality: Open-source ScienceQA’s state-of-the-art (SOTA) multimodal model was trained in just an hour, and while Google’s models still hold a slight quality advantage, the gap is closing rapidly.
Moreover, open-source models are faster, more customizable, more private, and achieve more with fewer resources. The author emphasizes the need for Google to consider its value add in this changing landscape, as users may not pay for restricted models when free, unrestricted alternatives offer comparable quality.
The document also recounts how the open-source community gained access to Meta’s LLaMA (a capable foundation model) and the subsequent outpouring of innovation, resulting in developments like instruction tuning, quantization, quality improvements, and human evaluations. This lowered the barrier to entry for training and experimentation, empowering individuals to contribute and innovate.
The current open-source renaissance in LLMs mirrors the earlier renaissance in image generation, which saw rapid innovation and widespread adoption, eventually outpacing the large players in the industry. The author believes that this could also be the case for LLMs, given the similar structural elements.
The document further highlights that Google missed opportunities to capitalize on these innovations. For example, Low Rank Adaptation (LoRA) is a powerful technique for reducing the cost and time of model fine-tuning, enabling rapid model personalization on consumer hardware. By focusing more on the work done by the open-source community, Google could avoid reinventing the wheel and embrace new approaches that directly address their current challenges.
The leaked document presents a compelling case for Google to reevaluate its position in the AI landscape, recognizing the significant advancements and potential of open-source AI models. As these models become more efficient and accessible, Google must adapt and collaborate with the broader AI community to maintain its competitive edge in the rapidly evolving world of artificial intelligence.
