The Llama 3 Herd of Models

2407.21783

Total: 1

#1 The Llama 3 Herd of Models [PDF⁹⁵] [Copy] [Kimi¹⁴⁰] [REL]

Authors: Jack Aaron Grattafiori, Jack Abhimanyu Dubey, Jack Abhinav Jauhri, Jack Abhinav Pandey, Jack Abhishek Kadian, Jack Ahmad Al-Dahle, Jack Aiesha Letman, Jack Akhil Mathur, Jack Alan Schelten, Jack Alex Vaughan, Jack Amy Yang, Jack Angela Fan, Jack Anirudh Goyal, Jack Anthony Hartshorn, Jack Aobo Yang, Jack Archi Mitra, Jack Archie Sravankumar, Jack Artem Korenev, Jack Arthur Hinsvark, Jack Arun Rao, Jack Aston Zhang, Jack Aurelien Rodriguez, Jack Austen Gregerson, Jack Ava Spataru, Jack Baptiste Roziere, Jack Bethany Biron, Jack Binh Tang, Jack Bobbie Chern, Jack Charlotte Caucheteux, Jack Chaya Nayak et al. (531 additional authors not shown)

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.

Subjects: Artificial Intelligence , Computation and Language , Computer Vision and Pattern Recognition

Publish: 2024-07-31 17:54:27 UTC

2407.21783

#1 The Llama 3 Herd of Models [PDF95] [Copy] [Kimi140] [REL]

#1 The Llama 3 Herd of Models [PDF⁹⁵] [Copy] [Kimi¹⁴⁰] [REL]