Meta, the company formerly known as Facebook, has released a new state-of-the-art large language model called LLaMA (Large Language Model Meta AI). This new foundational model is designed to help researchers advance their work in the field of artificial intelligence (AI), particularly in natural language processing (NLP). Smaller and more performant models like LLaMA enable others in the research community who do not have access to large amounts of infrastructure to study these models, which further democratizes access in this important, fast-changing field.
LLaMA is available in several sizes, including 7B, 13B, 33B, and 65B parameters, making it easier to retrain and fine-tune for specific potential product use cases. LLaMA was trained on text from the 20 languages with the most speakers, focusing on those with Latin and Cyrillic alphabets, making it more versatile and able to be applied to many different use cases.
Large language models like LLaMA have shown new capabilities in recent years, such as generating creative text, predicting protein structures, and answering reading comprehension questions. However, full research access to these models remains limited due to the resources required to train and run such large models. This restricted access has limited researchers’ ability to understand how and why these large language models work, hindering progress on efforts to improve their robustness and mitigate known issues, such as bias, toxicity, and the potential for generating misinformation.
To address these challenges, Meta has shared the code for LLaMA with other researchers to more easily test new approaches to limiting or eliminating these problems in large language models. The company also provides in the paper a set of evaluations on benchmarks evaluating model biases and toxicity to show the model’s limitations and to support further research in this crucial area.
In order to prevent misuse, access to the model will be granted on a case-by-case basis to academic researchers, those affiliated with organizations in government, civil society, and academia, and industry research laboratories around the world. Meta also believes that the entire AI community, including academic researchers, civil society, policymakers, and industry, must work together to develop clear guidelines around responsible AI in general and responsible large language models in particular.
LLaMA is a significant milestone in the development of large language models and NLP research, as it enables greater accessibility and understanding of these models. It is a promising step toward more responsible and inclusive AI development that can benefit billions of people worldwide.
