Meta has unveiled Llama 3.1 405B, a groundbreaking open-source AI model that sets a new standard for flexibility, control, and state-of-the-art capabilities. Llama 3.1 is designed to empower developers by offering unmatched performance in general knowledge, steerability, math, tool use, and multilingual translation. This release is part of Meta’s commitment to making AI accessible to everyone, supporting the community in unlocking new workflows such as synthetic data generation and model distillation, with an emphasis on safety and responsible use.
The Llama 3.1 405B model boasts a significantly expanded context length of 128K and supports eight languages, making it suitable for advanced use cases like long-form text summarization, multilingual conversational agents, and coding assistants. Meta has optimized the training process to achieve these results, using over 16,000 H100 GPUs to handle the immense computational load. The model’s architecture follows a standard decoder-only transformer model with iterative post-training procedures, ensuring high-quality synthetic data generation and improved performance across various capabilities.
Extensive evaluations of Llama 3.1 against competing models, including GPT-4 and Claude 3.5 Sonnet, demonstrate its competitive edge. The evaluations cover over 150 benchmark datasets and real-world scenarios, showing that Llama 3.1 excels in tasks such as general knowledge, coding, math, reasoning, and tool use. This performance is supported by a robust training infrastructure and careful data curation, ensuring both the quantity and quality of pre- and post-training data.
In addition to the 405B model, Meta has also upgraded its 8B and 70B models, enhancing their multilingual capabilities and extending their context length. These models are now available for download on llama.meta.com and Hugging Face, with immediate development support from a broad ecosystem of partner platforms. Meta’s Llama ecosystem includes tools like Llama Guard 3 and Prompt Guard for improved security and safety, as well as a reference system to help developers create custom agents and new types of agentic behaviors.
