Open-source giant Llama 3.1 emerges as a top performer and generates buzz
Under the attention of the tech community, Llama 3.1 series models stand out with their outstanding performance, especially its 405B super-large-cup version, which has demonstrated extraordinary strength in several reviews on Microsoft Azure-ML GitHub platform, not only surpassing GPT-4o, but even the 70B version can be separated from GPT-4o. It is worth noting that this is only the initial performance of the BASE model, and the INSTITUTION model, which has been trained with fine alignment, is expected to bring even more impressive score improvement.
However, a series of recent leaks have also added a bit of mystery to Llama 3.1. Download links, model cards with official score results, and detailed profiles were accidentally exposed, and although they have yet to be officially confirmed, they have already created a furor on the Internet, inspiring widespread discussion and anticipation in the industry.
If the leaked data is true, Llama 3.1 will undoubtedly become a leader in the open-source field and the entire AI macromodeling community, and its influence may surpass that of many existing closed-source flagship models.
Meanwhile, exciting news came from the ICML site, where Soumith Chintala, founder of PyTorch, formally announced in his speech that the Llama 3.1 series of models will be officially released on July 23rd (July 24th, Beijing time), injecting new vitality into the AI field.
Llama 3.1 Highlights Fast Facts：
- Multi-language Dialogue Optimization: instruct model is deeply optimized for multi-language scenarios, supporting multi-language text and code output to meet the needs of global users.
- Context Window Dramatically Expanded: The context window for each version of the model has surged from 8k to 128k, a 16-fold improvement, providing users with a smoother, more coherent dialog experience.
- Massive training resources: Llama 3.1 was trained on H100-80GB hardware using a cumulative total of 39 million GPU hours, of which the 405B version exclusively accounts for 31 million GPU hours, which ensures the robust performance of the model. The training data covers about 15 trillion tokens and incorporates a rich fine-tuned dataset.
With the revelation of Llama version 3.1, not only is the 405B mega-cup version highly anticipated, but the released 8B and 70B models will also receive upgrades.The Smol AI team has created a comparison table based on the leaked version of the model cards to visualize the performance improvements between the versions. Particularly impressive are the significant improvements on harder tasks in the 8B version, and the advantages in math and API calls in the 70B version.
In addition, Llama 3.1's 405B model weights take up a whopping 820GB of hard disk space, demonstrating its massive scale. Meanwhile, the model supports bf16 native precision and is expected to be available in an official FP8 quantized version to ease the deployment burden.
It is worth mentioning that the new open source protocol for Llama 3.1 brings significant changes, removing the restriction against using Llama 3 to improve other models and encouraging developers to innovate based on Llama. However, all models trained using Llama outputs are required to include "Llama" in their names to maintain brand consistency.
Facing a strong challenge from Llama 3.1, whether OpenAI will launch new products to defend its market position has become the center of attention in the industry. With the reactivation of Ottoman's personal account, this week may witness another round of revelry in the field of AI macromodeling.
Hi, I'm lcab-ljj, if you like my content then like to follow it ~ I will continue to present you with the most cutting-edge reports, but also welcome you to check out my other articles!/The GPT tutorials are available at/2024/07/16/Fomepay/