Meta Releases Llama 3

Meta released Llama 3 in April 2024 with 8-billion and 70-billion parameter models. The new generation used a tokenizer with a 128,000-token vocabulary and was trained on more than 15 trillion tokens, a large increase over Llama 2.

A larger and improved tokenizer can represent code and multilingual text more efficiently because common sequences require fewer tokens. Meta also invested heavily in filtering and preparing the training data rather than increasing parameter count alone.

Llama 3 strengthened the open-weight ecosystem. Developers could run the 8B model on relatively accessible hardware, deploy larger versions on their own servers, and fine-tune models without sending private data to a hosted API.